-- Logs begin at Fri 2022-07-29 04:24:37 UTC, end at Fri 2022-07-29 05:42:23 UTC. -- Jul 29 05:05:30.797234 dv-cinder-851279-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 29 05:05:30.806811 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 29 05:05:30 2022] *** Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: nodename: dv-cinder-851279-2-iscsi Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: machine: x86_64 Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: clock source: unix Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: pcre jit disabled Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: detected number of CPU cores: 4 Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: current working directory: / Jul 29 05:05:30.807801 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: detected binary path: /usr/bin/uwsgi-core Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: your processes number limit is 47680 Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: your memory page size is 4096 bytes Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: detected max file descriptor number: 2048 Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: lock engine: pthread robust mutexes Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: thunder lock: enabled Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 29 05:05:30.809002 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 29 05:05:30.847566 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: Python main interpreter initialized at 0x56521427a9c0 Jul 29 05:05:30.847566 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: python threads support enabled Jul 29 05:05:30.847566 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: your server socket listen backlog is limited to 100 connections Jul 29 05:05:30.847566 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: your mercy for graceful operations on workers is 90 seconds Jul 29 05:05:30.847949 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: mapped 403077 bytes (393 KB) for 2 cores Jul 29 05:05:30.848026 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: *** Operational MODE: preforking *** Jul 29 05:05:30.848785 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: *** uWSGI is running in multiple interpreter mode *** Jul 29 05:05:30.848785 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: spawned uWSGI master process (pid: 80159) Jul 29 05:05:30.849013 dv-cinder-851279-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 29 05:05:30.849978 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 1 (pid: 80169, cores: 1) Jul 29 05:05:30.849978 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 2 (pid: 80170, cores: 1) Jul 29 05:05:30.850129 dv-cinder-851279-2-iscsi devstack@keystone.service[80159]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 29 05:05:36.448220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 05:05:36.448220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 29 05:05:36.448991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 05:05:36.448991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 29 05:05:36.459883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 05:05:36.459883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jul 29 05:05:36.460363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 05:05:36.460363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jul 29 05:05:36.498508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 05:05:36.498688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 05:05:36.498873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 05:05:36.499141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 05:05:36.499318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 05:05:36.499598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.499844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.500033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.500217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.500397 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.500595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.500964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.501145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.501398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.501646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.501876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.502057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.502235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.502414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.502591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.502822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.503069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.503316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.503586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.503764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.503941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.504183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.504429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.504689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.504923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.505156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.505386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.505631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.505880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.506153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.506653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.506836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.507070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.507300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.507504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.507750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.507984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.508218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.508448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.508525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 05:05:36.508688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.508811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 05:05:36.508888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.508957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 05:05:36.509036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 05:05:36.509240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 05:05:36.509411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.509995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.510071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.510179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.510273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.510350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.510476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.510558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.510715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.510804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.510961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.511035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.511195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.511270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.511470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.511566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.511703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.511778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.511888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.511962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.512068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.512179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.512255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.512416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.512508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.512647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.512723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.512891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.512975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.513125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.513206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.513369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.513453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.513553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.513634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.513734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.513816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.513980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.514058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.514227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.514302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.514459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.514545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.514691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.514767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.514925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.515000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.515158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.515251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.515426 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.515505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.515675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.515761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.515953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.516038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.516186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.516416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.516492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.516625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.516709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.516858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.516935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.517088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.517170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.517270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.517348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.517517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.517594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.517753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.517836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.517989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.518070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.518222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.518305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.518466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.518541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.518688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.518760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.518828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.518920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.519003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.519137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.519220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.519316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.519422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.519576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 05:05:36.519662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.519826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.519912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.520057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.520619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.520804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.521985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.522933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.523960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.524887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.525850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.526989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.527959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.528980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.529930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.530850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.531971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.532975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.533920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.534871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.535832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.536416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.536639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.536873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.537877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.538995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.539983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.540939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.541920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.542971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.543925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.544866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.545995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.546939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.547939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.548960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.549926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.550878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.551031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.551110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.551255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.551334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.551602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.552930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.553912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.554997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.555835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556258 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.556968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.557922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.558958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.559936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.560877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.561866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.562995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.563996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.564968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.565999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.566934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.567119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.567198 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.567391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.567643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.567805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.567886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.568880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.569978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.570945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.571943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.572968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.573870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.574980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.575981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.576939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.577962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.578953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.579972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.580939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.581910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.582007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.582144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.582296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.582400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.582558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.583239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.583410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.583841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.583921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.584897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.585854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.586918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.587846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 05:05:36.587960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.588201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.588459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.588716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.588959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.589196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.589431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.589668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.589922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.590180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.590429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.590667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.590905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.591138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.591394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.591637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.591889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.592128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.592385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.592640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.592883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.593132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.593378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.593628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.593867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.594109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.594348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.594597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.594848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.595080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.595326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.595646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.595911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.596160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.596414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.596656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.596902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.597141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.597379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.597617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.597856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.598103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.599387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.599656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.599900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.600157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.600401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.600653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.600890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.601134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.601376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 05:05:36.601550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 05:05:36.617118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.617348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.617571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.625919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.626326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.626562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.626792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.635749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.655614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.655847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.665207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.665444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.708867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.709774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.759296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.759496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.760355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.760601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.824374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.824611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.824883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.825877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.826123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.849100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.849361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.849799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.849956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.850911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.851039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.851039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.851039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.861298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.861703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.874699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.874896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.875998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.876127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.876257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.876386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.876536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 05:05:36.893202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.893381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.893514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 05:05:36.893644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 05:05:36.893785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.893914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.894041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 05:05:36.894168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 05:05:36.894297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.894424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.894559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.894702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.894828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.894955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.895095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.895308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.904749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.904897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.905027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.905155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.905282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.905421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.905560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.905689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.913098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.913252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.917996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.918150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.918282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.918412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.918540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.918680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.918820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.918948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.919876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.920125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.920261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.920388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.920515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.920666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.920666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.944073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.944496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.945547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.945828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.946004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.946289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.950411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.950691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.950907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.950990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.951191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.951508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.957525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.957832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.958342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.958826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.961992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.962275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.962554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.962831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.963103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.963411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.963693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.963923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.964054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.964215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.964504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.964790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.965070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.965361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.965642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.965926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.980256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.980531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.982613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.982896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.984419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.986841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.987576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.987871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.990692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.990990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.992853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.995490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.996065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.998681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:36.999195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.999520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:36.999808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.000105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.000392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.000692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.001933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.002236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.002527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.002820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.003105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.003436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.016654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.016888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.017155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.017419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.019446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.019738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.020011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.020305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.024462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.026226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.026596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.027212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.028791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.029082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.029466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.029747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.030039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.030322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.030619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.031952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.032241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.032542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.032832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.033124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.044332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.044332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.044332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.044794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.044931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.045119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.045395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.046315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.046560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.046853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.047140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.047453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.047734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.048013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.061703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.064410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.064660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.067454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.067776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.070331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.070681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.071876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.072159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.072443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.073464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.074976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.075267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.075589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.078807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.080437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.082500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.083510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.083797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.084082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.084349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.084484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.087425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.087722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.088010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.088299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.092299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.092592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.092880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.096634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.096934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.097218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.100208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.100489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.100769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.101055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.101330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.101609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.101881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.102155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.102532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.102812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.103096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.103385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.103663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.103923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.104759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 05:05:37.105042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.105326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.105605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.105892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.106176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.106450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.106720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.107106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.107416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.107705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.107985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.108254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.108518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 05:05:37.137427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56521427a9c0 pid: 80170 (default app) Jul 29 05:05:37.140764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896393d-826a-44d8-b6ec-bac20cadf1e9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:37.141055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896393d-826a-44d8-b6ec-bac20cadf1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:37.141333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896393d-826a-44d8-b6ec-bac20cadf1e9 None None] PATH_INFO: `/v3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:37.143192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56521427a9c0 pid: 80169 (default app) Jul 29 05:05:37.146175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Fri Jul 29 05:05:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:44.855649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:44.856231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:44.856231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:44.860638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:05:44.866014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:44.866406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:44.866583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:44.936867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] Registered counter for database keystone Jul 29 05:05:44.958583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 05:05:44.964826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 05:05:45.155165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 05:05:45.155620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 05:05:45.213339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:45.215006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:45.345079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 29 05:05:45.495678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:45.566183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] Registered counter for database keystone Jul 29 05:05:45.586862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 05:05:45.593170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 05:05:45.718119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fc3ac-6095-4016-8a3e-e872474df830 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:45.722411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:45.723074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:45.724241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:45.724530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:45.733294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:45.787750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 05:05:45.788021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d57fcb-7ac3-4578-9ab1-e921b87b3fe2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:45.798240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Jul 29 05:05:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:05:45.805160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:45.820439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8bf03-f05a-488a-bf59-04917314510e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:45.824980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:45.826112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:45.826411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:45.826694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] PATH_INFO: `/v3/projects/3340905a228b4864923e8ce0f5cfd8a4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:45.828238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] RBAC: Authorizing `identity:get_project(project_id=3340905a228b4864923e8ce0f5cfd8a4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:45.879184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 05:05:45.880444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee154-ab02-4c5a-9998-e0a821c00902 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:45.884998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Fri Jul 29 05:05:45 2022] GET /identity/v3/projects/3340905a228b4864923e8ce0f5cfd8a4 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:47.898483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9cff1-6cf5-49ab-882e-52eda2b8c2a5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:47.898998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9cff1-6cf5-49ab-882e-52eda2b8c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:47.899080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9cff1-6cf5-49ab-882e-52eda2b8c2a5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:47.903170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:47.908481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:47.908925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:47.909052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:47.930048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:47.931616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:47.939621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 29 05:05:47.962160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9cff1-6cf5-49ab-882e-52eda2b8c2a5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:47.975241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9cff1-6cf5-49ab-882e-52eda2b8c2a5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:47.979549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c5730-0451-4845-a5a1-b1dd062f9f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:47.980171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c5730-0451-4845-a5a1-b1dd062f9f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:47.980466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c5730-0451-4845-a5a1-b1dd062f9f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:47.980750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c5730-0451-4845-a5a1-b1dd062f9f22 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:47.988566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c5730-0451-4845-a5a1-b1dd062f9f22 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:47.990416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c5730-0451-4845-a5a1-b1dd062f9f22 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:47.999311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Jul 29 05:05:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:05:48.005650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:48.014887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e2c3c-06c5-4129-ae12-95c68fad6bb9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:48.019277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06606e9-8b57-4304-b98b-1c5889c585a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:48.019998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06606e9-8b57-4304-b98b-1c5889c585a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:48.021321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06606e9-8b57-4304-b98b-1c5889c585a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:48.021612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06606e9-8b57-4304-b98b-1c5889c585a3 admin admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:48.023001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06606e9-8b57-4304-b98b-1c5889c585a3 admin admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:48.024891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06606e9-8b57-4304-b98b-1c5889c585a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:48.026451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Fri Jul 29 05:05:48 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:50.112388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa0617-b2a5-4f4c-bd60-00889faf2fd7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.112934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa0617-b2a5-4f4c-bd60-00889faf2fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.112934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa0617-b2a5-4f4c-bd60-00889faf2fd7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.114357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:50.119571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.120023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.120120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.139253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:50.140497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:50.156621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af8765-eda4-4c18-b8b4-ddd9dd7ce3f0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.157001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af8765-eda4-4c18-b8b4-ddd9dd7ce3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.157288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af8765-eda4-4c18-b8b4-ddd9dd7ce3f0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.158665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:05:50.164510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.164804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.165088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.206945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 05:05:50.207936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 05:05:50.224535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:50.246823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.253983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:50.256497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:50.272100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:50.354235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ade6e-d8da-4245-b61c-b1622c314043 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:50.358429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:05:50.380889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.389338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:50.390008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.390297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.390583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.403131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:05:50.413157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:05:50.414160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:05:50.414990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:05:50.420476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.486817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09c757-341b-4523-b3cf-7a05a5308e94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:50.489381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:50.505553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:50.524755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d6872-9223-49c4-8a97-d08ac53907ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:50.528296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:50.529178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.529469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.529751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.560068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:50.560702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.560988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.561266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.561987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:50.562296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:50.563735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:50.575755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:50.581342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.600317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:50.601241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:50.688681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc4bf9-69e2-40c6-9ad2-97a0b7e24290 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:50.702286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 05:05:50.708564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.711892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:50.817553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea9543-2a41-4c95-97c2-5b60db6bedcc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:50.831479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:50.832163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.832507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.832956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.834262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:50.835834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:50.837751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:50.843187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.855066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:50.855970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.856257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.856536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.858926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:50.876018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:50.877744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:05:50.898486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:05:50.899522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:05:50.900595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:05:50.900595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:05:50.900595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:05:50.900595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:05:50.900595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:05:50.905367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:50.949627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e1d9b-02c7-4573-a889-b4a0b00a7011 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:50.962909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:50.983234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:50.984008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:50.984298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:50.984574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:50.985820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:50.987304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:50.990492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:50.996406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:51.009401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c0172-e195-4037-aadb-3367959dd377 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:51.044122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:51.044754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:51.045039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:51.045317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:51.046004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:51.046314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:51.047745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b9b1b-e99d-4806-b52f-5cbacb10ed48 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:51.065611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:51.084590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:51.106965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea2ee-3e6f-45ed-ab92-d733a36d6a1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:51.144586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:51.145347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:51.145557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:51.145945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:51.146660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.domain.id=default, user.id=a74b2847a9d0472682c985005f47d70f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:51.146982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:51.149915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf339a-50cd-4e31-9809-58de95ee66ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:51.156459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Jul 29 05:05:50 2022] GET /identity/v3/role_assignments?user.id=a74b2847a9d0472682c985005f47d70f&scope.domain.id=default&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:53.510638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65840126-0453-42f5-bc93-4444f07c943d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.511115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65840126-0453-42f5-bc93-4444f07c943d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.511115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65840126-0453-42f5-bc93-4444f07c943d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.512693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:53.517904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526dd77-2617-4146-8360-f4a738e35a36 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.518304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526dd77-2617-4146-8360-f4a738e35a36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.518481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526dd77-2617-4146-8360-f4a738e35a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.548467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c526dd77-2617-4146-8360-f4a738e35a36 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:53.549714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c526dd77-2617-4146-8360-f4a738e35a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:53.642387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/23] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:53.677578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65840126-0453-42f5-bc93-4444f07c943d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:53.762991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bad607-a5e6-4f9d-badf-836a4a617666 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.763611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bad607-a5e6-4f9d-badf-836a4a617666 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.764124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bad607-a5e6-4f9d-badf-836a4a617666 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.767430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:53.772948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19207d-8694-497d-8ba1-5ba5650ece67 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.773576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19207d-8694-497d-8ba1-5ba5650ece67 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.774084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19207d-8694-497d-8ba1-5ba5650ece67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.799133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ad19207d-8694-497d-8ba1-5ba5650ece67 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:53.800906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad19207d-8694-497d-8ba1-5ba5650ece67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:53.840980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65840126-0453-42f5-bc93-4444f07c943d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:53.864405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:53.909566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26e22-f2a2-46ee-9ed4-64c87d7393d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:53.910310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26e22-f2a2-46ee-9ed4-64c87d7393d1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.910721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26e22-f2a2-46ee-9ed4-64c87d7393d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.911009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26e22-f2a2-46ee-9ed4-64c87d7393d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.938575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26e22-f2a2-46ee-9ed4-64c87d7393d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-86e26e22-f2a2-46ee-9ed4-64c87d7393d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:05:53.955767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:05:53.956713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:05:53.957605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/25] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:05:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:05:53.960989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/26] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:05:53.961159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946921-7c1a-403a-bfa0-fbf8969ca683 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.961459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946921-7c1a-403a-bfa0-fbf8969ca683 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.961744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946921-7c1a-403a-bfa0-fbf8969ca683 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.963173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/27] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:53.969122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:53.969421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:53.969702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:53.976951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946921-7c1a-403a-bfa0-fbf8969ca683 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:53.977485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:53.996866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:53.998248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:54.124811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/28] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:54.125453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:54.268035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e47e5-82a0-49da-805f-daa9c4fed394 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:54.283337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946921-7c1a-403a-bfa0-fbf8969ca683 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:54.308953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:54.309982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.310451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.310921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.311861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:54.313520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:54.317829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:54.329747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:54.330927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.331268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.331582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.332425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:54.332818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:54.334923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20824a-36b4-4700-835e-efe037f7b6f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:54.349310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/29] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:05:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:54.350482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674544a4-a3bf-4252-b7dc-ac35a288d63c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.350781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674544a4-a3bf-4252-b7dc-ac35a288d63c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.351059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674544a4-a3bf-4252-b7dc-ac35a288d63c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.352560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/30] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:54.353267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674544a4-a3bf-4252-b7dc-ac35a288d63c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:54.403708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:54.458001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/31] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 05:05:54 2022] POST /identity/v3/roles => generated 219 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:05:54.459357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:54.478267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674544a4-a3bf-4252-b7dc-ac35a288d63c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:54.527997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b403207-477a-405c-8b4d-643908809cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:54.528640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b403207-477a-405c-8b4d-643908809cd6 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.528922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b403207-477a-405c-8b4d-643908809cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.529207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b403207-477a-405c-8b4d-643908809cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.529947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b403207-477a-405c-8b4d-643908809cd6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:54.532223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b403207-477a-405c-8b4d-643908809cd6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:54.673639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f7f26-fdba-4228-92d7-16a0174b4342 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:54.711707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/32] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 05:05:54 2022] POST /identity/v3/roles => generated 225 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:05:54.713751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.714051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.714339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.724365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f57c7-ca48-48a1-8f0d-678af32274f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:54.725472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f57c7-ca48-48a1-8f0d-678af32274f7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.726846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f57c7-ca48-48a1-8f0d-678af32274f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.727501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f57c7-ca48-48a1-8f0d-678af32274f7 None admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.729274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f57c7-ca48-48a1-8f0d-678af32274f7 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:54.731282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f57c7-ca48-48a1-8f0d-678af32274f7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:54.733346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:05:54 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:54.734967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae79b3a-c102-474d-8a23-a9c5b22adfe7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.735452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae79b3a-c102-474d-8a23-a9c5b22adfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.735874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae79b3a-c102-474d-8a23-a9c5b22adfe7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.737400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/34] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:54.746508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:54.747116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:54.747614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:54.749322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:54.751252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:54.771745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:54.773871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:54.854621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/35] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:54.855733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:54.871753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:54.891780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/36] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:54.894307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:54.917960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:55.036127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccf331-f9f3-4bfb-b81e-2a1bacdffe41 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.037781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fc55b-a200-4545-952d-418bcaa7cd3e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.084191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.084864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.085152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.085433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.086906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.088645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.094984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/37] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:55.094984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:55.100973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.101843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.102221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.102686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.118775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.124355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:05:55.142533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:05:55.143460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:05:55.144254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:05:55.144254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:05:55.144254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:05:55.144254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:05:55.144254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/38] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:05:55.145577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:55.205952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:55.227459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:55.233907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a76d3-882c-4683-b0c4-846c183aba51 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.266740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888c89c-95ac-47fa-8fc1-847a70433bff None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.275248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.275932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.276227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.276515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.277246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.278660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.294681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/39] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 05:05:55 2022] POST /identity/v3/roles => generated 223 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:05:55.295485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:55.307048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b33eb7-7a59-4488-b34a-bbfc694d438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.308101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b33eb7-7a59-4488-b34a-bbfc694d438d None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.309112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b33eb7-7a59-4488-b34a-bbfc694d438d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.309587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b33eb7-7a59-4488-b34a-bbfc694d438d None admin] PATH_INFO: `/v3/domains/default/users/a74b2847a9d0472682c985005f47d70f/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.312947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b33eb7-7a59-4488-b34a-bbfc694d438d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a74b2847a9d0472682c985005f47d70f, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.316088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b33eb7-7a59-4488-b34a-bbfc694d438d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.344291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/40] 10.222.0.49 () {58 vars in 1361 bytes} [Fri Jul 29 05:05:55 2022] PUT /identity/v3/domains/default/users/a74b2847a9d0472682c985005f47d70f/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:05:55.344291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7efa-6ce3-4084-b3b4-9b1e2d50d085 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.344291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7efa-6ce3-4084-b3b4-9b1e2d50d085 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.344291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7efa-6ce3-4084-b3b4-9b1e2d50d085 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.344291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/41] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:55.349931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.349931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.350191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.372139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:55.373582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:55.461464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6156a0-3e60-49f3-a6e2-e5c26683e290 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.474885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/42] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:55.499917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:55.512794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.514402 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.514905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.515471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.516487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:55.517516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.519074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.535822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/43] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:55.542530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:55.548417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:55.564256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:55.646424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16c8b-76ec-4b0c-933f-c0955d3e6e2b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.682866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b532a7-b65e-4f01-b791-80129446cd5c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.700104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.700773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.701063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.701343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.702512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.704041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.706016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/44] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:55.711668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:55.727585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff10330-7de7-4e7d-9b93-6ed46dcdb563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.728618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff10330-7de7-4e7d-9b93-6ed46dcdb563 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.729114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff10330-7de7-4e7d-9b93-6ed46dcdb563 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.729587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff10330-7de7-4e7d-9b93-6ed46dcdb563 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.730853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff10330-7de7-4e7d-9b93-6ed46dcdb563 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.733372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff10330-7de7-4e7d-9b93-6ed46dcdb563 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.788139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/45] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:05:55 2022] POST /identity/v3/projects => generated 301 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:05:55.856696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c72b3-5946-474c-9241-49d868e93a7d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:55.858702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:55.903755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517718-abee-421c-9c40-f27a3facc2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:55.904414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517718-abee-421c-9c40-f27a3facc2e5 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:55.904711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517718-abee-421c-9c40-f27a3facc2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:55.905000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517718-abee-421c-9c40-f27a3facc2e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:55.905729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28517718-abee-421c-9c40-f27a3facc2e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:55.907297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28517718-abee-421c-9c40-f27a3facc2e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:55.946666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/46] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:05:55 2022] POST /identity/v3/projects => generated 312 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:05:57.434347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02348d80-e9eb-4e2f-8d75-831b790c2307 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:57.434922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02348d80-e9eb-4e2f-8d75-831b790c2307 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:57.434922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02348d80-e9eb-4e2f-8d75-831b790c2307 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:57.436444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/47] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:05:57.442063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:57.442508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:57.442647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:57.461692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:57.462841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:57.535219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/48] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:05:57.560702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02348d80-e9eb-4e2f-8d75-831b790c2307 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:57.630917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:57.669546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02348d80-e9eb-4e2f-8d75-831b790c2307 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:57.705884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:57.706552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:57.706846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:57.707134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:57.718714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:05:57.724133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:05:57.725017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:05:57.725765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/49] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:05:57.731239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:57.822437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:57.844648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2c1cd-4634-48af-8179-eb2bc3b17461 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:57.878370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:57.879006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:57.879846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:57.880134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:57.880864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:57.881174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:57.882577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76870-9ed6-48fb-830e-f1eff191ab51 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:57.890912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/50] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:57.897145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:57.997102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d27afb-6ebc-4106-98cc-90d8eb5ce91d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:58.021092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15621ac-cd34-4a68-8529-eba1bc7c7a9b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.021461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15621ac-cd34-4a68-8529-eba1bc7c7a9b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.021657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15621ac-cd34-4a68-8529-eba1bc7c7a9b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.023065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/51] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:05:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:05:58.025766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:58.028790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.029093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.029388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.034531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:58.035217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.035631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.035923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.037402 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:58.038899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:58.040621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/52] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:05:57 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:05:58.045983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:58.047927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:05:58.049161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:05:58.119648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/53] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:05:58.142609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:58.151399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389136b-c19b-4a84-84fb-c8974e1c9167 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:58.190195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:58.190853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.191144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.191457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.193261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:58.194866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:58.196562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/54] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:58.202386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:58.233971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:58.259405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f219b7a-4982-4d06-92f0-e700fb128c36 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:58.265974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:58.293876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:58.294542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.294830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.295115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.296334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:58.297808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:58.299447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/55] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:05:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:58.306162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:05:58.307432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae849ad0-c8ca-48b7-9436-a7b4a50a155d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:58.343516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:58.344159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.344448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.344725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.345528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.domain.id=default, user.id=a74b2847a9d0472682c985005f47d70f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:05:58.345845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:58.347366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5d152-d83a-4e6c-b39a-0fa7191080ec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:58.357099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/56] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Jul 29 05:05:58 2022] GET /identity/v3/role_assignments?user.id=a74b2847a9d0472682c985005f47d70f&scope.domain.id=default&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:05:58.419833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687378-bbd6-4d18-ac01-82b7fbc0fb53 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:05:58.461137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e2d1d-d1c8-4f17-9457-58f88e8214b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:05:58.461853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e2d1d-d1c8-4f17-9457-58f88e8214b7 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:05:58.462080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e2d1d-d1c8-4f17-9457-58f88e8214b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:05:58.462362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e2d1d-d1c8-4f17-9457-58f88e8214b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:05:58.463076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e2d1d-d1c8-4f17-9457-58f88e8214b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:05:58.464652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e2d1d-d1c8-4f17-9457-58f88e8214b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:05:58.482012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:05:58.495962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/57] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Jul 29 05:05:58 2022] POST /identity/v3/projects => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:06:00.544996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437798ab-013f-41db-bc69-081f576ef413 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:00.545503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437798ab-013f-41db-bc69-081f576ef413 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:00.545503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437798ab-013f-41db-bc69-081f576ef413 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:00.546864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/58] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:00.553043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:00.553470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:00.554470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:00.574574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:00.575805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:00.641419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/59] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:00.665829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437798ab-013f-41db-bc69-081f576ef413 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:00.769780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437798ab-013f-41db-bc69-081f576ef413 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:00.796231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:00.805212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d5fd-9a8f-4091-af20-3e67d52baf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:00.805891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d5fd-9a8f-4091-af20-3e67d52baf4f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:00.806185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d5fd-9a8f-4091-af20-3e67d52baf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:00.806473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d5fd-9a8f-4091-af20-3e67d52baf4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:00.807771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91d5fd-9a8f-4091-af20-3e67d52baf4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:00.809301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91d5fd-9a8f-4091-af20-3e67d52baf4f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:00.811005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/60] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:00.817151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:00.920580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db427c5-e195-4b89-a40b-00997df01500 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:00.946674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:00.955954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80539352-d2f0-4946-82d7-68d40a8e553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:00.956597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80539352-d2f0-4946-82d7-68d40a8e553e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:00.956895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80539352-d2f0-4946-82d7-68d40a8e553e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:00.957181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80539352-d2f0-4946-82d7-68d40a8e553e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:00.957917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80539352-d2f0-4946-82d7-68d40a8e553e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:00.959437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80539352-d2f0-4946-82d7-68d40a8e553e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:00.987288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/61] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:00 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:04.258537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ddf27-288e-4ce4-a44b-925ab0e2f3d2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.259841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ddf27-288e-4ce4-a44b-925ab0e2f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.260513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ddf27-288e-4ce4-a44b-925ab0e2f3d2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.263495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/62] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:04.269706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.271526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.272143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.313036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:04.315136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:04.366897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5438c-a2d9-4976-be5f-9a726a5f3483 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.367708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5438c-a2d9-4976-be5f-9a726a5f3483 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.368352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5438c-a2d9-4976-be5f-9a726a5f3483 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.370985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/63] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:04.376411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.377066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.377570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.413409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:04.415743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:04.481569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/64] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:04.506214 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:04.571455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/65] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:04.607258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:04.744360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cba6df-7fbd-40ea-a92e-694e068b7627 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:04.778468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:04.828311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8434-fdfc-4fdc-9baa-96b5eaaaffed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:04.831949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa81d4-f461-4599-8d83-3bb82fac9dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:04.833319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa81d4-f461-4599-8d83-3bb82fac9dfc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.833615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa81d4-f461-4599-8d83-3bb82fac9dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.833899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa81d4-f461-4599-8d83-3bb82fac9dfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.845642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa81d4-f461-4599-8d83-3bb82fac9dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-81aa81d4-f461-4599-8d83-3bb82fac9dfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:04.852027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:04.853662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:04.855225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:04.859716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/66] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:04.860096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8356c0-8e79-4a65-afb3-60444ca4e329 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.860395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8356c0-8e79-4a65-afb3-60444ca4e329 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.860683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8356c0-8e79-4a65-afb3-60444ca4e329 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.862347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/67] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:04.867941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8356c0-8e79-4a65-afb3-60444ca4e329 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:04.881951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:04.892998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce319fac-ff7f-4b89-8d70-882f42d0c046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:04.893661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319fac-ff7f-4b89-8d70-882f42d0c046 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.893948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319fac-ff7f-4b89-8d70-882f42d0c046 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.894228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319fac-ff7f-4b89-8d70-882f42d0c046 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.906651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce319fac-ff7f-4b89-8d70-882f42d0c046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ce319fac-ff7f-4b89-8d70-882f42d0c046 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:04.912687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:04.915982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/68] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:04.919440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:04.922202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:04.922202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:04.939313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:04.940675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:05.038102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/69] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:05.042286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.042768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8356c0-8e79-4a65-afb3-60444ca4e329 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.084962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.085186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.085323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.085607 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.086360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:05.086676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.088186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbf365-23a8-4f79-91ff-186978303c71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.101025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/70] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:05.102163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d7ed-85f9-43ea-8e6d-0650bb91093f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.102465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d7ed-85f9-43ea-8e6d-0650bb91093f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.102754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d7ed-85f9-43ea-8e6d-0650bb91093f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.104409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/71] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:05.105655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5b283-6a88-4bee-b208-ac0e81724c83 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.105865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5b283-6a88-4bee-b208-ac0e81724c83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.106287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5b283-6a88-4bee-b208-ac0e81724c83 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.107592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/72] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:05.108458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5b283-6a88-4bee-b208-ac0e81724c83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.150089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.171825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96c0d5-fd92-4c28-857d-2f6cefcc1428 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.197885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.219045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.219725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.220017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.220297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.221021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:05.221339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.222781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.229305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5b283-6a88-4bee-b208-ac0e81724c83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.234009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/73] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:05.234009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.271497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611b1f0-5ba2-4ae5-b7d6-22bbd4bb8119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.272050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611b1f0-5ba2-4ae5-b7d6-22bbd4bb8119 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.272193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611b1f0-5ba2-4ae5-b7d6-22bbd4bb8119 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.272466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611b1f0-5ba2-4ae5-b7d6-22bbd4bb8119 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.283571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611b1f0-5ba2-4ae5-b7d6-22bbd4bb8119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0611b1f0-5ba2-4ae5-b7d6-22bbd4bb8119 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:05.289669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:05.291300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/74] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:05.292508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0902b-f5a6-4a64-b1f1-1ca9cd2dfd58 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.293423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0902b-f5a6-4a64-b1f1-1ca9cd2dfd58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.293423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0902b-f5a6-4a64-b1f1-1ca9cd2dfd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.311503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1d0902b-f5a6-4a64-b1f1-1ca9cd2dfd58 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:05.312733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1d0902b-f5a6-4a64-b1f1-1ca9cd2dfd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:05.363846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c55da6-b76e-49d0-a9eb-78a7449d56a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.396805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/75] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:05.396805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.396805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.396805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.417185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:05.418446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:05.421517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.422180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.422467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.422745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.424004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.425441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.427930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/76] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:05.428732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.445229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.486811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.499167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/77] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:05.500120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.553259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5b12-c8b9-4221-b176-140de484afd1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.596461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.597125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.597413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.597718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.613087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.615124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.616834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/78] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:05.617068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594de3-7cd0-45a1-9653-8ffc91d468da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.617614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.661571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.662211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.662502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.662787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.663547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:05.663862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.665268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.675434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/79] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:05.676641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.697004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.727094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31849bb4-0f58-4072-8047-caa04b337fd4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.729901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.767975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.768631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.768924 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.769207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.770933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.772470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.774093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/80] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:05.774882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.792197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e542d-6b4f-4c28-87a1-7b65d0be44fc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.828030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.828584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.828875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.829158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.875486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451696ff-5086-44fd-9bc1-47f0f4d16c39 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:05.898352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:05.899895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:05.910194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:05.910943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:05.911235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:05.911556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:05.916860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:05.923235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:05.929082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:05.930096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:05.930845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/81] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:05.931554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:05.967136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.006152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/82] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:06.007233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:06.042533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe5200-aca5-4dba-854c-6ad2e5fa13f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.076492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.077121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.077406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.077683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.108108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203ca5e-9c1b-4e47-9563-aa084c4bb877 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.126150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.145483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.146152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.146438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.146720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.147963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.149462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.150825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.151125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/83] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:06 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:06.151937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:06.152444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.193654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.275433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/84] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:06.275433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:06.298947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9ebe5-c8c5-4206-ada6-9c7b6a686f4d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.335871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.336505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.336791 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.337073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.337772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.339274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.366937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/85] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:06 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:06.368931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:06.418045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9991073-8a69-4278-91aa-eb723f883a94 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.456806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.465018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.466002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.466380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.466691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.467068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.467880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:06.468208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.469666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.481915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/86] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:06.483618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:06.586769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1da78-10c6-423b-9dde-23394a5bcdde None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.640840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eca843-bb60-4e4a-9f82-3b48dd69f984 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.653846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.654649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.655015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.655905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.657377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.659629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.667622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/87] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:06 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:06.667622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:06.712933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.714243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.714837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.715605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.821991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1c88-a104-4e24-b5f9-3282cff8da17 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:06.831646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.838684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:06.852653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.854518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.885630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:06.886325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:06.886615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:06.886898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:06.888523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:06.890075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:06.892032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/88] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:06 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:06.892806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:07.059608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b4dae-6e50-4f34-b379-42e37d370b68 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:07.068200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/89] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:07.070280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:07.121292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:07.123151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:07.124103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:07.125358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:07.129241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:07.133080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:07.138061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/90] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:06 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:07.139875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:07.203974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:07.269199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:07.273014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f6fc0-1cd2-450d-925d-1af5cd327664 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:07.286688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d3fc1-506f-4cc4-b0a4-35d4cce77f8e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:07.315492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:07.316340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:07.316788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:07.317446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:07.318524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=98034ca1d05b4b12b4b8ee5fe315e059) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:07.318964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:07.320676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:07.327148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/91] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:07 2022] GET /identity/v3/role_assignments?user.id=98034ca1d05b4b12b4b8ee5fe315e059&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=f5c39945c9694134955532988101506f => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:07.336573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:07.369552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:07.371111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:07.371945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:07.372640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:07.374375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=a74b2847a9d0472682c985005f47d70f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:07.375456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:07.378804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:07.391292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/92] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:07 2022] GET /identity/v3/role_assignments?user.id=a74b2847a9d0472682c985005f47d70f&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:07.393385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:07.571682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733436ca-e594-4b67-bc7a-e48bc80609e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:07.592415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:07.620842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49520d-6099-453a-8622-7a7ed8aec247 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:07.648316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:07.649062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:07.649358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:07.649645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:07.651158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:07.652849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:07.654701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/93] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:07 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:07.675733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:07.710370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:07.712244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:07.712722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:07.713197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:07.783957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:07.889996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:07.893035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:08.017381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e9a3d-ac63-4424-b259-8895d75cc294 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:08.090634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/94] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:08.099736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:08.132099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:08.133210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:08.133714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:08.134213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:08.135778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] RBAC: Adding query filter params (role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=98034ca1d05b4b12b4b8ee5fe315e059) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:08.136309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:08.139033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:08.150715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/95] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:07 2022] GET /identity/v3/role_assignments?user.id=98034ca1d05b4b12b4b8ee5fe315e059&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 259 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:08.286376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:08.384523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4128e5-8692-4697-b606-a850348745c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:08.458100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d00e2-0850-4fc6-a927-906b42d2e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:08.459743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d00e2-0850-4fc6-a927-906b42d2e419 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:08.460530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d00e2-0850-4fc6-a927-906b42d2e419 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:08.461317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d00e2-0850-4fc6-a927-906b42d2e419 None admin] PATH_INFO: `/v3/projects/04d79c6096804cdbba49014487959176` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:08.463931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d00e2-0850-4fc6-a927-906b42d2e419 None admin] RBAC: Authorizing `identity:get_project(project_id=04d79c6096804cdbba49014487959176)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:08.467080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d00e2-0850-4fc6-a927-906b42d2e419 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:08.470653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/96] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:08 2022] GET /identity/v3/projects/04d79c6096804cdbba49014487959176 => generated 312 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:08.480989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:08.595341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:08.730929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4816f3-38a4-46bf-9664-5b5f54c25d2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:08.805673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:08.806730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:08.807238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:08.807763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:08.809267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, scope.project.id=04d79c6096804cdbba49014487959176, user.id=98034ca1d05b4b12b4b8ee5fe315e059) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:08.809839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:08.812737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0b2c-a22a-4e71-914b-6e35dcced21f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:08.829431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/97] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:08 2022] GET /identity/v3/role_assignments?user.id=98034ca1d05b4b12b4b8ee5fe315e059&scope.project.id=04d79c6096804cdbba49014487959176&role.id=f5c39945c9694134955532988101506f => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:10.262631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab2d0-7e1a-4a6b-99f2-43324aad8c1c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:10.263652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab2d0-7e1a-4a6b-99f2-43324aad8c1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:10.264173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab2d0-7e1a-4a6b-99f2-43324aad8c1c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:10.265849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/98] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:06:10.287785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:10.288300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:10.288770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:10.325778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:10.327828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:10.459292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ed168-9738-4c5b-8125-da66aab79973 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:10.459578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ed168-9738-4c5b-8125-da66aab79973 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:10.459935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ed168-9738-4c5b-8125-da66aab79973 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:10.461392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/99] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:06:10.472243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:10.472861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:10.472861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:10.485532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/100] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:10.512584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:10.513892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:10.579430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:10.649235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/101] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:10.704039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:10.739657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:10.905063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8b0df-b964-4fe1-adc7-a8de92d4b6cf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:10.907889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:11.013938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:11.015453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.016262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.016987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.019537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:11.023741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:11.027853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/102] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:11.037614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:11.093497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518ef15-a196-4c48-8104-2b991c1fc771 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:11.174709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:11.175428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.176255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.176596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.209133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:11.218016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:11.219694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:11.221127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/103] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:11.238659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:11.401239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c6a39-6a55-41b5-87a3-fd57a197f60b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:11.404429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:11.414446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fecbd-721b-4d6f-a635-f784e5b6808d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:11.456104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:11.482802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:11.483871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.485346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.487181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.488164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:11.490084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:11.492910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094cc1d-2ef7-4637-8d3c-d30da5641b4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:11.505421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:11.506569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.507146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.507674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.508615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:11.510308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:11.523389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/104] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:11.523389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a7c6c-65cd-4fca-b471-8183347429f3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.523389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a7c6c-65cd-4fca-b471-8183347429f3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.523683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a7c6c-65cd-4fca-b471-8183347429f3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.526091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/105] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:11.535935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409077-6b0e-4f4e-95e1-1e57d3b29fdb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.535935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409077-6b0e-4f4e-95e1-1e57d3b29fdb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.536626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409077-6b0e-4f4e-95e1-1e57d3b29fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.550484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/106] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:06:11 2022] POST /identity/v3/projects => generated 302 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:11.552141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:11.601447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6d409077-6b0e-4f4e-95e1-1e57d3b29fdb None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:11.604197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6d409077-6b0e-4f4e-95e1-1e57d3b29fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:11.687313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25483a8-c6b0-4fa8-a55c-471ccc4588cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:11.736673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:11.737732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.739069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.739708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] PATH_INFO: `/v3/users/f950bbf752914b2fb153b035fbab2a55` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.765275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] RBAC: Authorizing `identity:get_user(user_id=f950bbf752914b2fb153b035fbab2a55)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:11.768344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:11.772023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/107] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:11 2022] GET /identity/v3/users/f950bbf752914b2fb153b035fbab2a55 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:11.780186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:11.804520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:11.866681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/108] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:11.892943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef59e3a-1353-41bb-9d6b-b2428b141bc1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.893451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef59e3a-1353-41bb-9d6b-b2428b141bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.893953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef59e3a-1353-41bb-9d6b-b2428b141bc1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.897395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/109] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:11.914314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.914895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.915268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.919751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a2ba8-3381-4aed-b59e-57afabd1ff07 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:11.960894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:11.961902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:11.962437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:11.962928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:11.965746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:11.968460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:12.063572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:12.066493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:12.089664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:12.124242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/110] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:12.125471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:12.225863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:12.306358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78f964-045d-49fd-a691-ca1b057b7227 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:12.342329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/111] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 05:06:12.348194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:12.368224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:12.369269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.369761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.370241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.392518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:12.399604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:12.400504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:12.401264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:12.409639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/112] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:12.411674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:12.505832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490194d7-3c05-4f31-b54f-73cbff078852 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:12.525253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:12.552685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:12.554040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.554597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.555150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.568915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:12.575590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:12.576594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:12.577348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:12.579842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/113] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:12.580952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:12.716362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc220af-96d1-48da-a4f4-e678cf8d0283 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:12.736864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:12.745164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14d17-644c-4bb0-ad2e-84aec3d8321f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:12.759923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:12.760470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.760764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.761049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.761791 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:12.762157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:12.763803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:12.782873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/114] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:12.782873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:12.822883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c438a26-df78-423f-a116-7b1b0682fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:12.824124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c438a26-df78-423f-a116-7b1b0682fb66 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.824960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c438a26-df78-423f-a116-7b1b0682fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.825550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c438a26-df78-423f-a116-7b1b0682fb66 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.828649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c438a26-df78-423f-a116-7b1b0682fb66 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:12.831701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c438a26-df78-423f-a116-7b1b0682fb66 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:12.835336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/115] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:12 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:12.836058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:12.842491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b624f7a-b833-4fd9-8a10-4ed546dcb236 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.843019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b624f7a-b833-4fd9-8a10-4ed546dcb236 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.848059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b624f7a-b833-4fd9-8a10-4ed546dcb236 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.851147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/116] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:12.853292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee81e1-ce4c-41bb-84ab-6298cbd78857 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.853814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee81e1-ce4c-41bb-84ab-6298cbd78857 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.854329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee81e1-ce4c-41bb-84ab-6298cbd78857 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.857880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/117] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:12.858893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee81e1-ce4c-41bb-84ab-6298cbd78857 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:12.940669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69126414-39c3-4ae3-9e23-8b344b453361 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:12.986521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:12.987599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:12.988031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:12.988540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:12.989587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:12.990043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:12.992023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:13.003235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/118] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:13.004812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.067995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee81e1-ce4c-41bb-84ab-6298cbd78857 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:13.128636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de864-63da-41c0-9cfb-07fc359b42d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:13.130050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de864-63da-41c0-9cfb-07fc359b42d1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.130500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de864-63da-41c0-9cfb-07fc359b42d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.130904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de864-63da-41c0-9cfb-07fc359b42d1 None admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.132611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de864-63da-41c0-9cfb-07fc359b42d1 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:13.134366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de864-63da-41c0-9cfb-07fc359b42d1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:13.136237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/119] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:12 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:13.141127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.141645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.142146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.164066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:13.166665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:13.183493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:13.188994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067772d6-f408-4d0f-9eba-fbb93aa0bfb3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:13.215571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:13.257827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:13.258668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.259248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.259643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.260574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] RBAC: Adding query filter params (role.id=902e6af348874520b982b3f959494f04, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=f950bbf752914b2fb153b035fbab2a55) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:13.260984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:13.262730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b6383-f3c6-4261-bff3-a62e6d7271dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:13.270746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/120] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:13 2022] GET /identity/v3/role_assignments?user.id=f950bbf752914b2fb153b035fbab2a55&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=902e6af348874520b982b3f959494f04 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:13.273186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.273709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.274199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.296045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/121] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:13.297944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.301653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:13.303335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:13.402978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/122] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:13.404514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.513957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149d971-3803-451f-9d19-54e56c7e4e1d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:13.549973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:13.571900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:13.572605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.572866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.573154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.573651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd08747-da16-4864-bb1b-d349b64e728c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:13.574652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:13.576301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:13.578794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/123] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:13 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:13.583525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.631901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:13.632609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.632949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.633196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.635502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:13.637031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:13.638851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/124] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:13 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:13.644654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.654313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:13.761236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79301f-c09c-4616-9747-28bd95deff97 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:13.808582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:13.809255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.809544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.809830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.823636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:13.829816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:13.832101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:13.835530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/125] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:13.840246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.852878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c04c5-9a21-453a-b280-18fdae125677 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:13.899758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:13.900907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:13.901425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:13.901990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:13.915426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:13.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:13.922326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:13.923136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:13.926640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/126] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:13.932749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:13.961695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:14.063788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:14.070847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5184f-5a1a-4733-a655-8ce0d151837b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.083335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55286-f9ac-4077-b45e-e101a0874382 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.130966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.131700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.131995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.132284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/a74b2847a9d0472682c985005f47d70f/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.134730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c486928a4e4e68b08704afe7cd789d, user_id=a74b2847a9d0472682c985005f47d70f, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.137561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.149282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.150077 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.150356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.150639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.152251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.154036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.155997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/127] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:13 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:14.156885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.178434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/128] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:13 2022] PUT /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/a74b2847a9d0472682c985005f47d70f/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:14.180521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.288729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa51e8b-3b5a-4fdd-8dd3-6b828c0a7537 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.330359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.330359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.330797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.330922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.331699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:14.332097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.333552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.347478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/129] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:14.347478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.362507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:14.382240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:14.424780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9402f-9d64-45d7-94fd-19241b539873 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.473239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bcad7-37bf-4058-bd96-dc30fecc1047 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.508002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.509604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.510682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.511726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.513648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.514248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:14.514533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.514830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.515117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/98034ca1d05b4b12b4b8ee5fe315e059/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.516038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.517283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c486928a4e4e68b08704afe7cd789d, user_id=98034ca1d05b4b12b4b8ee5fe315e059, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.518869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.520271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.539165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/130] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:14 2022] PUT /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/98034ca1d05b4b12b4b8ee5fe315e059/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:14.539906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/131] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:14.551697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.554429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.688202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:14.732408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17056f-565b-4e37-8b50-abca2ae0fae5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.782609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.783663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.783663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.783944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.785477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.787006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.789010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/132] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:14 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:14.791877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c2cac-4d72-409b-a4c3-654685db1e60 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.794581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.865846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.867806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.868677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.869480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.871883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.873805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.877620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/133] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:14 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:14.887889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:14.909296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:14.926706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae046e-2a7c-487d-b770-dc6160b0ec2d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:14.967096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73beb2-3dae-4c28-97b3-a1f985e801d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:14.968213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73beb2-3dae-4c28-97b3-a1f985e801d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.968733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73beb2-3dae-4c28-97b3-a1f985e801d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.969231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73beb2-3dae-4c28-97b3-a1f985e801d6 None admin] PATH_INFO: `/v3/projects/04d79c6096804cdbba49014487959176` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.970764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73beb2-3dae-4c28-97b3-a1f985e801d6 None admin] RBAC: Authorizing `identity:get_project(project_id=04d79c6096804cdbba49014487959176)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:14.973344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73beb2-3dae-4c28-97b3-a1f985e801d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:14.975952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/134] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:14 2022] GET /identity/v3/projects/04d79c6096804cdbba49014487959176 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:14.977457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dd551-fa7e-44f1-b844-f2830eec9ce8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:14.978014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dd551-fa7e-44f1-b844-f2830eec9ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:14.978515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dd551-fa7e-44f1-b844-f2830eec9ce8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:14.984817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/135] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:14.987082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dd551-fa7e-44f1-b844-f2830eec9ce8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:15.011756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:15.106384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dd551-fa7e-44f1-b844-f2830eec9ce8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:15.110205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4464c-3d67-485a-a980-4cb91b7c8820 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:15.147976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8839789-da09-4d9d-9508-5d58a4fce71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:15.149086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8839789-da09-4d9d-9508-5d58a4fce71f None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.149633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8839789-da09-4d9d-9508-5d58a4fce71f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.150116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8839789-da09-4d9d-9508-5d58a4fce71f None admin] PATH_INFO: `/v3/projects/04d79c6096804cdbba49014487959176/users/98034ca1d05b4b12b4b8ee5fe315e059/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.152536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8839789-da09-4d9d-9508-5d58a4fce71f None admin] RBAC: Authorizing `identity:create_grant(project_id=04d79c6096804cdbba49014487959176, user_id=98034ca1d05b4b12b4b8ee5fe315e059, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:15.154256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8839789-da09-4d9d-9508-5d58a4fce71f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:15.170150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/136] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:14 2022] PUT /identity/v3/projects/04d79c6096804cdbba49014487959176/users/98034ca1d05b4b12b4b8ee5fe315e059/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:06:15.172209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.172764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.173287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.195486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:15.197178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:15.198974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:15.200399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.200995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.201809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.205226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:15.207852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:15.211916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/137] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:14 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:15.222441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:15.282672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/138] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:15.380911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:15.410373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:15.411301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6990ae-78d6-4820-8b40-befe7a6c50e8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:15.446116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:15.450502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:15.451224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.451564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.451986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/98034ca1d05b4b12b4b8ee5fe315e059/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.454340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c486928a4e4e68b08704afe7cd789d, user_id=98034ca1d05b4b12b4b8ee5fe315e059, role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:15.456047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:15.478233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/139] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:15 2022] PUT /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/98034ca1d05b4b12b4b8ee5fe315e059/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:15.665656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51bafb-8dc0-44e8-b71d-8e071d34d228 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:15.733620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531b0e6-0175-4bc8-b850-7d7e367ea717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:15.735059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531b0e6-0175-4bc8-b850-7d7e367ea717 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.735645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531b0e6-0175-4bc8-b850-7d7e367ea717 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.736136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531b0e6-0175-4bc8-b850-7d7e367ea717 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.737679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531b0e6-0175-4bc8-b850-7d7e367ea717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:15.740052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531b0e6-0175-4bc8-b850-7d7e367ea717 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:15.743231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/140] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:15.749182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:15.904469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e21cb-7a75-4951-a6ce-4693c82b47ba None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.906045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e21cb-7a75-4951-a6ce-4693c82b47ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.906830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e21cb-7a75-4951-a6ce-4693c82b47ba None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.913255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/141] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:15.917245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:15.918072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:15.918868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:15.920972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4b0bf-0cb1-4bc5-bb13-d0c59ae07cbb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:15.969586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:15.972341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:16.007836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:16.008828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:16.009273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:16.009702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:16.010826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:16.013247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:16.039940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:16.085014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/142] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:15 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:16.128543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/143] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:16.156796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:16.476208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4a02-d4fc-492f-a0b0-a86f2ce29406 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:16.533820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:16.534557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:16.534905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:16.535754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:16.550438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:16.562074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:16.563684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:16.565183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/144] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:16.572295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:16.586872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:16.713891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef5ab6-3f70-4f20-b7de-da53efc23540 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:16.756655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:16.757228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:16.757522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:16.757811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:16.758553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:16.758869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:16.760335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:16.770903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/145] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:16.777072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:16.853960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:16.928481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943362f-862d-4851-ab4a-819f2174fc86 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:16.978830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:16.979687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:16.979984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:16.980316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] PATH_INFO: `/v3/users/f950bbf752914b2fb153b035fbab2a55` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:16.981707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] RBAC: Authorizing `identity:get_user(user_id=f950bbf752914b2fb153b035fbab2a55)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:16.983249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:16.986043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/146] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:16 2022] GET /identity/v3/users/f950bbf752914b2fb153b035fbab2a55 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:16.991453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:17.095274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:17.135829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18662a-ca73-4369-a7a5-d0f593553131 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:17.179755 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c52e9c-2219-4b7c-a474-d4bcaf3477ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:17.180849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52e9c-2219-4b7c-a474-d4bcaf3477ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:17.181341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52e9c-2219-4b7c-a474-d4bcaf3477ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:17.181819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52e9c-2219-4b7c-a474-d4bcaf3477ac None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:17.183334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c52e9c-2219-4b7c-a474-d4bcaf3477ac None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:17.185117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c52e9c-2219-4b7c-a474-d4bcaf3477ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:17.187076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/147] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:16 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:17.194383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:17.328583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064892-b305-414c-8f63-9281f6f6753d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:17.355061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:17.374475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dff1a-0ee6-4386-a03d-fc92dc816860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:17.375584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dff1a-0ee6-4386-a03d-fc92dc816860 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:17.376078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dff1a-0ee6-4386-a03d-fc92dc816860 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:17.376582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dff1a-0ee6-4386-a03d-fc92dc816860 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/f950bbf752914b2fb153b035fbab2a55/roles/902e6af348874520b982b3f959494f04` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:17.379535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dff1a-0ee6-4386-a03d-fc92dc816860 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c486928a4e4e68b08704afe7cd789d, user_id=f950bbf752914b2fb153b035fbab2a55, role_id=902e6af348874520b982b3f959494f04)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:17.381385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dff1a-0ee6-4386-a03d-fc92dc816860 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:17.399411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/148] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:17 2022] PUT /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d/users/f950bbf752914b2fb153b035fbab2a55/roles/902e6af348874520b982b3f959494f04 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:19.015640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c5d3b-30fd-4f83-914e-e525dbe57efe None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.016664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c5d3b-30fd-4f83-914e-e525dbe57efe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.017192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c5d3b-30fd-4f83-914e-e525dbe57efe None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.018797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/149] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:19.025222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007c550-25a7-4d98-9d30-f862567c2c52 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.026222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007c550-25a7-4d98-9d30-f862567c2c52 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.026720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007c550-25a7-4d98-9d30-f862567c2c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.056448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5007c550-25a7-4d98-9d30-f862567c2c52 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:19.058006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5007c550-25a7-4d98-9d30-f862567c2c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:19.071879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08ac46-e900-47a8-bf8f-1c853228bb39 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.072642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08ac46-e900-47a8-bf8f-1c853228bb39 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.073134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08ac46-e900-47a8-bf8f-1c853228bb39 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.074754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/150] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:19.080172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.080752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.081253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.125199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:19.127841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:19.228297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/151] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:19.229944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9527-ac0c-4cfd-8d17-546d65cde2c0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.230438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9527-ac0c-4cfd-8d17-546d65cde2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.230939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9527-ac0c-4cfd-8d17-546d65cde2c0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.232617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/152] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:19.238713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0fac2-91a5-4481-ae39-4b005ade2009 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.244006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0fac2-91a5-4481-ae39-4b005ade2009 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.244456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0fac2-91a5-4481-ae39-4b005ade2009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.266098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/153] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:19.274647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:19.316765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d9a0fac2-91a5-4481-ae39-4b005ade2009 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:19.318824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9a0fac2-91a5-4481-ae39-4b005ade2009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:19.392930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:19.423818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c58d3-a304-458c-925f-dcdbac911760 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:19.464274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:19.465364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.465952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.466507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.479973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:19.486353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:19.487214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:19.488066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:19.490873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/154] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:19.491936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:19.525822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/155] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:19.527341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac21d-5f43-417c-bca0-3b757be970cd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.527824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac21d-5f43-417c-bca0-3b757be970cd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.528232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac21d-5f43-417c-bca0-3b757be970cd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.529672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/156] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:19.530518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac21d-5f43-417c-bca0-3b757be970cd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:19.610908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98adad-03a6-46f9-9785-5cf5472e8d9d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:19.631685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:19.659378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:19.666631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b71d51-877b-4c81-bba5-c5a9f9446264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:19.667693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b71d51-877b-4c81-bba5-c5a9f9446264 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.668182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b71d51-877b-4c81-bba5-c5a9f9446264 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.668656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b71d51-877b-4c81-bba5-c5a9f9446264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.687883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b71d51-877b-4c81-bba5-c5a9f9446264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-62b71d51-877b-4c81-bba5-c5a9f9446264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:19.701627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:19.702572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:19.703639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:19.706115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/157] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:19.708804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689eb43-895d-4520-8628-690281ae7518 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.709354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689eb43-895d-4520-8628-690281ae7518 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.709847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689eb43-895d-4520-8628-690281ae7518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.752461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5689eb43-895d-4520-8628-690281ae7518 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:19.754299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5689eb43-895d-4520-8628-690281ae7518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:19.757051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac21d-5f43-417c-bca0-3b757be970cd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:19.828312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:19.831371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:19.831371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:19.831371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:19.831371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:19.831371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:19.832226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:19.847168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/158] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:19.848635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:19.892097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/159] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:19.893416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689eb43-895d-4520-8628-690281ae7518 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:20.037617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fef8a2-96ad-41f2-9b9e-7c9fed5b95d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:20.039572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:20.114259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689eb43-895d-4520-8628-690281ae7518 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:20.128186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:20.128939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.129305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.129599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.152446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:20.164572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:20.170176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:20.171088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:20.171975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:20.179543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/160] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:20.180606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:20.246885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:20.247997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.248707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.249216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.250587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:20.251155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:20.254907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:20.277291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/161] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:20.278174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:20.460675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68252b6c-80de-4e8c-8994-aa4f6f3d46c3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:20.489271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5b73e-1d9e-4932-8891-5ed229abb967 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:20.525914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:20.526936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.527041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:20.527588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.527792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.528057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.528343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.528644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.529779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:20.540074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:20.542356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/162] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:20 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:20.548204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:20.550114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:20.559066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:20.560025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:20.561044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:20.568841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/163] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:20.570297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:20.648034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:20.703752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:20.827768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51252-5f5d-4d18-92b5-99468dbd7855 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:20.869876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac40f5-c9fd-4a03-a621-c3246584cc98 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:20.948112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:20.951052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.952296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.952787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.954083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:20.954438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc13c-f311-47cc-9f5e-7160a4495859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:20.954694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:20.955122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc13c-f311-47cc-9f5e-7160a4495859 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.955456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc13c-f311-47cc-9f5e-7160a4495859 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.955745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc13c-f311-47cc-9f5e-7160a4495859 None admin] PATH_INFO: `/v3/users/a74b2847a9d0472682c985005f47d70f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.957648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:20.957760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc13c-f311-47cc-9f5e-7160a4495859 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b2847a9d0472682c985005f47d70f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:20.959325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc13c-f311-47cc-9f5e-7160a4495859 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:20.961697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/164] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:20 2022] GET /identity/v3/users/a74b2847a9d0472682c985005f47d70f => generated 247 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:20.963574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12f1fb-ab76-4e81-aa2b-0eb3cd88686e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:20.963872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12f1fb-ab76-4e81-aa2b-0eb3cd88686e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:20.964157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12f1fb-ab76-4e81-aa2b-0eb3cd88686e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:20.965876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/165] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:20.966738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12f1fb-ab76-4e81-aa2b-0eb3cd88686e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:20.975705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/166] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:20.977267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:21.101357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12f1fb-ab76-4e81-aa2b-0eb3cd88686e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:21.117489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:21.154065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:21.164852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c846d5-c881-48b9-92e1-633ef83284ed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:21.186226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:21.186881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.187191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.187499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.214453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:21.215095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.215406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.215687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.216761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:21.216917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:21.218280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62687f-3988-41cd-814a-e86f99711a6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:21.230410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/167] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:21.231941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138e6d7-3053-4214-af63-9d81213ea979 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.232241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138e6d7-3053-4214-af63-9d81213ea979 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.232534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138e6d7-3053-4214-af63-9d81213ea979 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.234382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/168] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:21.240211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.240421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.240712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.260459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:21.261694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:21.337067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/169] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:21.338027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:21.349872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:21.354550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:21.440125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:21.459872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197295-d4f0-4acb-856a-ca76419e1371 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:21.500667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b169a5-3260-4660-8df3-f3ec2cb945fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:21.501303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b169a5-3260-4660-8df3-f3ec2cb945fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.501592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b169a5-3260-4660-8df3-f3ec2cb945fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.501874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b169a5-3260-4660-8df3-f3ec2cb945fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.538174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/170] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:21.539042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:21.548030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:21.592982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b169a5-3260-4660-8df3-f3ec2cb945fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:21.594918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b169a5-3260-4660-8df3-f3ec2cb945fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:21.669099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3fc98-a59d-4bd4-b653-d21eac8af227 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:21.713414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:21.722248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:21.723209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.723613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.723957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.725543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:21.727461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:21.729654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/171] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:21 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:21.733795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:21.737160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/172] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:21.740991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.741517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.742026 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.764526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:21.766088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:21.838356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:21.856558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/173] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:21.860891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:21.869205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d51f10-f83e-42a2-8cca-57c9354691cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:21.907678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc3a3c-1d82-4bb5-92f7-38f9c3cc91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:21.908345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3a3c-1d82-4bb5-92f7-38f9c3cc91c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.908634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3a3c-1d82-4bb5-92f7-38f9c3cc91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.908914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3a3c-1d82-4bb5-92f7-38f9c3cc91c5 None admin] PATH_INFO: `/v3/users/98034ca1d05b4b12b4b8ee5fe315e059` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.910183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc3a3c-1d82-4bb5-92f7-38f9c3cc91c5 None admin] RBAC: Authorizing `identity:get_user(user_id=98034ca1d05b4b12b4b8ee5fe315e059)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:21.911708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc3a3c-1d82-4bb5-92f7-38f9c3cc91c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:21.913363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/174] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:21 2022] GET /identity/v3/users/98034ca1d05b4b12b4b8ee5fe315e059 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:21.914565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5752b-5398-47ba-86dd-613028fc732f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.914856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5752b-5398-47ba-86dd-613028fc732f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.915136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5752b-5398-47ba-86dd-613028fc732f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.916449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/175] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:21.917514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1419e-f385-4885-b0b5-76dbb287e5df None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:21.917805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1419e-f385-4885-b0b5-76dbb287e5df None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:21.918086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1419e-f385-4885-b0b5-76dbb287e5df None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:21.919637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/176] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:21.920163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1419e-f385-4885-b0b5-76dbb287e5df None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:21.979231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3523ea9-970d-4529-a78f-e3416a6b2be6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:21.994799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.017271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.017939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.018229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.018507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.030871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:22.036160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:22.037843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/177] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:22.039302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.041627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1419e-f385-4885-b0b5-76dbb287e5df None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.080511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.081285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.081810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.082649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.084157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.085874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.096113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/178] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:21 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:22.096113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.105749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.157769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faba70-3f86-494f-8ed0-073dad4e3873 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.204366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.205016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.205312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.205589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.212471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852bb97-b68d-492e-b5eb-3dbde87b518b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.261953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.262990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.263408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.263812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.265731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.265960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.268259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.270830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/179] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:22.271663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.302186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.304017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.360968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.385023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6dc18-e3e2-484d-acff-fb375fba3d82 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.419055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/180] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:22.419961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.425298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939bfe6-cbe6-4808-8310-fd59f41a9468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.426009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939bfe6-cbe6-4808-8310-fd59f41a9468 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.426300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939bfe6-cbe6-4808-8310-fd59f41a9468 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.426583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939bfe6-cbe6-4808-8310-fd59f41a9468 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.437763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939bfe6-cbe6-4808-8310-fd59f41a9468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7939bfe6-cbe6-4808-8310-fd59f41a9468 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:22.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:22.445370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:22.446927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:22.449918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/181] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:22.451111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717933ad-af6f-43b6-845e-3725e1990b35 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.451435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717933ad-af6f-43b6-845e-3725e1990b35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.451724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717933ad-af6f-43b6-845e-3725e1990b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.471435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-717933ad-af6f-43b6-845e-3725e1990b35 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:22.473246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-717933ad-af6f-43b6-845e-3725e1990b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:22.505530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.532653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f24b-d695-4915-ae67-7c9620f24e64 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.550835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/182] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:22.552258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.552811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.553090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.573430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:22.574675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:22.588216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.588864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.589154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.589430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.644397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.651504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/183] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:22.651504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.680822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.683074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.773641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2629d-4a18-41ad-adc0-ebfc9b46abdc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.775671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.813219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/184] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:22.814842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.820665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.821311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.821606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.821887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.822617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:22.822933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.824379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.833348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/185] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:22.834203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.898183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:22.937542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886cab9-6b12-4f36-bf98-86cf1979a9b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.944082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8b477-d3f8-493b-8f0e-28ab357adbb3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:22.975479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.976381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.976881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.977312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.978283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=98034ca1d05b4b12b4b8ee5fe315e059) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:22.978691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.980424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.981029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:22.981654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:22.982048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:22.982327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:22.983117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=a74b2847a9d0472682c985005f47d70f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:22.983458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:22.984899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:22.987765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/186] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/role_assignments?user.id=98034ca1d05b4b12b4b8ee5fe315e059&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=f5c39945c9694134955532988101506f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:22.992044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/187] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/role_assignments?user.id=a74b2847a9d0472682c985005f47d70f&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:22.992281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:22.993338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.078184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.174064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfbb0-db16-4bd0-9c17-e2840c84088e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.212731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9290036-c3da-4a2f-bfa6-f150c2296d81 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.238939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.239751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.240037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.240321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.243262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.243495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.246306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.248535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/188] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.249448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.272238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.273264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.273744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.275151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.275151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:23.276435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.278775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.297050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/189] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.300439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.391010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2bdf0-6eba-44b5-9694-acb845a65013 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.400457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.431871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.432539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.432828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.433118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.436048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc48abf-cc99-46fc-bdd0-9aa6ca65eb9d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.445471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:23.451823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:23.453219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/190] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:23.454695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.481696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.482362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.482740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.482933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.484195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.485707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.487378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/191] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.488763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.559951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.593817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bc318-e294-4b28-959d-39032c38f50b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.630988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6550d-b6a8-4d34-9921-c4b8a102d500 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.641738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.642332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.642613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.642898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] PATH_INFO: `/v3/projects/04d79c6096804cdbba49014487959176` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.644182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] RBAC: Authorizing `identity:get_project(project_id=04d79c6096804cdbba49014487959176)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.645862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.647826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/192] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/projects/04d79c6096804cdbba49014487959176 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.648432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.674062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.674725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.675020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.675298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] PATH_INFO: `/v3/users/969b5d44f7e149db82bd279593ff8129` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.687628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.691591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] RBAC: Authorizing `identity:get_user(user_id=969b5d44f7e149db82bd279593ff8129)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.693101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.694869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/193] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/users/969b5d44f7e149db82bd279593ff8129 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.695719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.767323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cdd45-e5d8-42f9-bc68-824f84160c02 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.805957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.806759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.807134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.807561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.808555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] RBAC: Adding query filter params (role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=98034ca1d05b4b12b4b8ee5fe315e059) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:23.808962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.810618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.811963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0a986-14e7-404a-8224-9116d96ec91f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:23.817890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/194] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/role_assignments?user.id=98034ca1d05b4b12b4b8ee5fe315e059&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.818822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.831011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.860538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:23.861286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:23.861580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:23.861866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] PATH_INFO: `/v3/users/f950bbf752914b2fb153b035fbab2a55` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:23.864805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=f950bbf752914b2fb153b035fbab2a55)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:23.867385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:23.870431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/195] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/users/f950bbf752914b2fb153b035fbab2a55 => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:23.872031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:23.953073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:23.958559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc1049-d810-4dfc-9b1e-5ebb93e97ff3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:24.024766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:24.025701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:24.026167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:24.026634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:24.027856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:24.028372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:24.030846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:24.045010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/196] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:24.046028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:24.062738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf136f-d07f-442b-8ee9-45db98661ec8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:24.107782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:24.108527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:24.108821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:24.109106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:24.109846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:24.111450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:24.143939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/197] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:23 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:24.145902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:24.193654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf6a59-baf0-4435-83db-276a732f5bfc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:24.201153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:24.268745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:24.269899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:24.270537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:24.271128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:24.272733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, scope.project.id=04d79c6096804cdbba49014487959176, user.id=98034ca1d05b4b12b4b8ee5fe315e059) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:24.273313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:24.276162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:24.282004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28bf8-cd94-4116-93cf-9b924dd9d342 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:24.295203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/198] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:24 2022] GET /identity/v3/role_assignments?user.id=98034ca1d05b4b12b4b8ee5fe315e059&scope.project.id=04d79c6096804cdbba49014487959176&role.id=f5c39945c9694134955532988101506f => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:24.295203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:24.330140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:24.364092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:24.364842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:24.365132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:24.365463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:24.477932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e86e2-f6bc-41aa-8dda-621393055132 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:24.492809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:24.494351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:24.534755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:24.537243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:24.537243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:24.537243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:24.547824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:24.634472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/199] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:24.635447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:24.679982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:24.683392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:24.733022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:24.784071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d952e8-0224-4586-a0e4-cb788d85912f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:24.824355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:24.824355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:24.824355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:24.824898 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] PATH_INFO: `/v3/users/969b5d44f7e149db82bd279593ff8129` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:24.825795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] RBAC: Authorizing `identity:get_user(user_id=969b5d44f7e149db82bd279593ff8129)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:24.827342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:24.835124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/200] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:24 2022] GET /identity/v3/users/969b5d44f7e149db82bd279593ff8129 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:24.835124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:24.840008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:24.897728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/201] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:24.899863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:24.982082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5507c-80a1-4d68-9892-0322e18a70aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:25.040721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:25.043514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:25.044403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:25.045358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:25.049092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:25.053160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:25.058746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/202] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:24 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:25.061558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:25.082448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f8464-13b4-499b-9af3-004896a0e84a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:25.114712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:25.129772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:25.130428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:25.130787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:25.131132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:25.216172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:25.252909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f3c46-2c01-496c-bec4-4bbf4893052d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:25.280719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:25.285338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:25.295055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:25.296398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:25.296913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:25.297366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:25.298800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:25.300617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:25.302939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/203] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:25 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:25.304021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:25.453080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/204] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:25.455006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:25.509249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:25.526949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c4d1d-fcf0-4651-90d3-9dcd22eda5a1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:25.576092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:25.576736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:25.577027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:25.577307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:25.578120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=969b5d44f7e149db82bd279593ff8129) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:25.578438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:25.580140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:25.588337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/205] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:25 2022] GET /identity/v3/role_assignments?user.id=969b5d44f7e149db82bd279593ff8129&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:25.589333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:25.593552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:25.650038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589136d3-b9f9-41da-9b87-aafd673f6819 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:25.697986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:25.698759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:25.699054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:25.699342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:25.700299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] RBAC: Adding query filter params (role.id=902e6af348874520b982b3f959494f04, scope.project.id=d5c486928a4e4e68b08704afe7cd789d, user.id=f950bbf752914b2fb153b035fbab2a55) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:25.700626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:25.702132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:25.717377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/206] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:25 2022] GET /identity/v3/role_assignments?user.id=f950bbf752914b2fb153b035fbab2a55&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=902e6af348874520b982b3f959494f04 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:25.842293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8214cac-07f0-4b9f-aabe-b4b781f765b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:25.915958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639969f-2c67-4c61-9c5e-3b211c3b5991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:25.917019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639969f-2c67-4c61-9c5e-3b211c3b5991 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:25.917532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639969f-2c67-4c61-9c5e-3b211c3b5991 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:25.918034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639969f-2c67-4c61-9c5e-3b211c3b5991 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:25.919588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639969f-2c67-4c61-9c5e-3b211c3b5991 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:25.921385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639969f-2c67-4c61-9c5e-3b211c3b5991 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:25.927666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/207] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:25 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:25.933507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:26.059709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:26.078906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fca0e6-76a2-4c2f-a783-1f9299f07ac7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:26.123894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:26.124844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:26.125280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:26.125703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:26.126645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] RBAC: Adding query filter params (role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=969b5d44f7e149db82bd279593ff8129) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:26.127122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:26.128868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e6fa-1e84-464d-8a38-1097e36d92e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:26.136011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/208] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:25 2022] GET /identity/v3/role_assignments?user.id=969b5d44f7e149db82bd279593ff8129&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:26.558359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11046c-ab30-4329-8fab-505222c5096c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:26.558921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11046c-ab30-4329-8fab-505222c5096c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:26.558921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11046c-ab30-4329-8fab-505222c5096c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:26.560785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/209] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:26.567143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:26.567671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:26.567745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:26.588389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:26.589606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:26.664089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/210] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:26.688219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11046c-ab30-4329-8fab-505222c5096c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:26.773661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:26.811742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11046c-ab30-4329-8fab-505222c5096c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:26.855012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813385bd-6896-4cad-a983-1aad6cd68ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:26.855717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813385bd-6896-4cad-a983-1aad6cd68ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:26.856004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813385bd-6896-4cad-a983-1aad6cd68ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:26.856284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813385bd-6896-4cad-a983-1aad6cd68ab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:26.868272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813385bd-6896-4cad-a983-1aad6cd68ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-813385bd-6896-4cad-a983-1aad6cd68ab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:26.873391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:26.874256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:26.875031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/211] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:26.882004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:26.948961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4456c0-5acb-4019-ab62-50142f9ec4a0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:26.949251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4456c0-5acb-4019-ab62-50142f9ec4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:26.949682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4456c0-5acb-4019-ab62-50142f9ec4a0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:26.952653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/212] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:26.957739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355a95-8189-4234-9010-c9b349d1864b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:26.958074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355a95-8189-4234-9010-c9b349d1864b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:26.958302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355a95-8189-4234-9010-c9b349d1864b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:26.979485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:26.981527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96355a95-8189-4234-9010-c9b349d1864b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:26.983059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96355a95-8189-4234-9010-c9b349d1864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:27.002323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8e9ab-f8ca-4117-a125-a40b82eb5234 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:27.044433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:27.045474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.046011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.046487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.047457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:27.047990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:27.049906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:27.061055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/213] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:27.067705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:27.087517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/214] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:27.107442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96355a95-8189-4234-9010-c9b349d1864b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:27.180849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:27.196944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cea03-d2b3-4477-8c28-b8954da7b50e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:27.235280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96355a95-8189-4234-9010-c9b349d1864b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:27.237310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:27.238483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.239857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.239857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] PATH_INFO: `/v3/users/adb56f48867f40cbb4aa45dc38449ef2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.255580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] RBAC: Authorizing `identity:get_user(user_id=adb56f48867f40cbb4aa45dc38449ef2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:27.257158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:27.259136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/215] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:27 2022] GET /identity/v3/users/adb56f48867f40cbb4aa45dc38449ef2 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:27.264721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:27.267515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:27.283822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:27.285531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.286439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.287032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:27.292454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:27.301000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/216] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:27.303029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:27.414038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f04c-0386-42bf-bdbc-13f3b841e20b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:27.426510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e435-25a3-4aea-9e34-4528807522b7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:27.463116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497710f-6b0f-41df-80bc-26f9b29ee394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:27.463797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497710f-6b0f-41df-80bc-26f9b29ee394 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.464082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497710f-6b0f-41df-80bc-26f9b29ee394 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.464364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497710f-6b0f-41df-80bc-26f9b29ee394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.465511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:27.475469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:27.476601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.477264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.478004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.478988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:27.480918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:27.513289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/217] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:27 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:27.564832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497710f-6b0f-41df-80bc-26f9b29ee394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:27.567024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497710f-6b0f-41df-80bc-26f9b29ee394 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:27.569301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:27.702859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/218] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 05:06:27.710643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:27.786582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4f1c-cce5-4966-830b-3f91f1f21f7a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.787029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4f1c-cce5-4966-830b-3f91f1f21f7a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.787511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4f1c-cce5-4966-830b-3f91f1f21f7a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.790365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/219] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:27.797652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.798078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.798486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.837804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd5c21-570f-4ab7-81e1-5041795551cc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:27.838514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:27.839790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:27.903239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:27.903513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:27.903872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:27.904267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:27.906345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:27.907986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:27.910369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/220] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:27 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:27.916403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:27.957013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/221] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:27.982121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:28.003273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:28.028030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:28.075852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad034a8-9115-4848-a22e-ea563dd00b07 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:28.130996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d10783-0606-4f45-ab05-c598b20951d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:28.148977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:28.149966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.150477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.150961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.151996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=adb56f48867f40cbb4aa45dc38449ef2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:28.152582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:28.154273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:28.165223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/222] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:27 2022] GET /identity/v3/role_assignments?user.id=adb56f48867f40cbb4aa45dc38449ef2&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=f5c39945c9694134955532988101506f => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:28.206078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:28.207474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.207678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.208048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.219325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:28.225435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:28.226384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:28.227161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/223] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:28.232951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:28.385944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81008105-2a54-4dad-8edf-af4e0878c5c2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:28.410623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:28.428856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:28.429507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.429796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.430079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.430816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:28.431124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:28.432569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c080d-8b17-43e9-89e8-10ce6f60aa64 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:28.442765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/224] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:28.448295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:28.565867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc2d2d-cb8b-433b-8a08-b6ea9981ab11 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.566272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc2d2d-cb8b-433b-8a08-b6ea9981ab11 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.566652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc2d2d-cb8b-433b-8a08-b6ea9981ab11 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.568045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/225] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:28.574602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.574888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.575163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.597776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:28.599577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:28.635714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:28.649179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d08078-c547-4e84-9541-64c4ed0ab79d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:28.723799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:28.724531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.724821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.725104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] PATH_INFO: `/v3/users/969b5d44f7e149db82bd279593ff8129` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.726738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] RBAC: Authorizing `identity:get_user(user_id=969b5d44f7e149db82bd279593ff8129)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:28.728275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:28.731385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/226] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:28.731872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/227] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:28 2022] GET /identity/v3/users/969b5d44f7e149db82bd279593ff8129 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:28.743656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:28.757339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:28.886127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba051d-00ec-4a11-a389-d083004d1549 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:28.923538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:28.924198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:28.924482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:28.924761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:28.937080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:28.942405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:28.943314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:28.944134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:28.948597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/228] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:28.949806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:28.961088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ae96a-1f39-4067-bc85-1f947b75238f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:28.974761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:28.979503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:29.023517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:29.024238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:29.024650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:29.024982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:29.026166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:29.027701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:29.029839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/229] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:28 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:29.035901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:29.068848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592be36-d818-4d15-a3e6-9961f4686f0b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:29.109860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:29.110656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:29.111012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:29.111298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:29.115378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:29.115378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:29.115378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:29.129922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/230] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:29.140913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:29.241321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388aeb9-96bd-4bcf-aa30-25caf1ae5167 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:29.283764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:29.296706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:29.297803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:29.298273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:29.298733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252/users/969b5d44f7e149db82bd279593ff8129/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:29.301955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] RBAC: Authorizing `identity:create_grant(project_id=faf7faecfc1b448388b564fbe426b252, user_id=969b5d44f7e149db82bd279593ff8129, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:29.303944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:29.304620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19171c57-ecde-4c74-80e2-6c32e4604934 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:29.343039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:29.343842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:29.344130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:29.344457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] PATH_INFO: `/v3/users/969b5d44f7e149db82bd279593ff8129` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:29.345668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] RBAC: Authorizing `identity:get_user(user_id=969b5d44f7e149db82bd279593ff8129)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:29.347923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:29.348586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/231] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:29 2022] PUT /identity/v3/projects/faf7faecfc1b448388b564fbe426b252/users/969b5d44f7e149db82bd279593ff8129/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:06:29.351382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/232] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:29 2022] GET /identity/v3/users/969b5d44f7e149db82bd279593ff8129 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:29.355721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:29.408913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:29.486331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfe87c-5f23-4a41-9a29-73ee26e71c53 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:29.533938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93cbf7-6694-4cd0-897b-a0f3c14c8cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:29.535374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93cbf7-6694-4cd0-897b-a0f3c14c8cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:29.535801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93cbf7-6694-4cd0-897b-a0f3c14c8cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:29.536489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93cbf7-6694-4cd0-897b-a0f3c14c8cb4 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:29.537933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93cbf7-6694-4cd0-897b-a0f3c14c8cb4 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:29.539648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93cbf7-6694-4cd0-897b-a0f3c14c8cb4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:29.542022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/233] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:29 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:29.548463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:29.673305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfae5a-e272-4b81-b900-21d4a58a2d0f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:29.711136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2f5a4-be67-45fd-8b4b-744b5ecf7dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:29.712098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2f5a4-be67-45fd-8b4b-744b5ecf7dfc None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:29.712524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2f5a4-be67-45fd-8b4b-744b5ecf7dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:29.712964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2f5a4-be67-45fd-8b4b-744b5ecf7dfc None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252/users/969b5d44f7e149db82bd279593ff8129/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:29.715296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2f5a4-be67-45fd-8b4b-744b5ecf7dfc None admin] RBAC: Authorizing `identity:create_grant(project_id=faf7faecfc1b448388b564fbe426b252, user_id=969b5d44f7e149db82bd279593ff8129, role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:29.716913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2f5a4-be67-45fd-8b4b-744b5ecf7dfc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:29.735244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/234] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:29 2022] PUT /identity/v3/projects/faf7faecfc1b448388b564fbe426b252/users/969b5d44f7e149db82bd279593ff8129/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:30.146992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4923-22a8-44b6-a139-c2a7fa0fe45f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.148068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4923-22a8-44b6-a139-c2a7fa0fe45f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.148570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4923-22a8-44b6-a139-c2a7fa0fe45f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.150595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/235] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:30.155833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f4672-b3b0-4e42-929c-9cb792bee354 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.156406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f4672-b3b0-4e42-929c-9cb792bee354 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.157049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f4672-b3b0-4e42-929c-9cb792bee354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.178016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a85f4672-b3b0-4e42-929c-9cb792bee354 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:30.179608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a85f4672-b3b0-4e42-929c-9cb792bee354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:30.182867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:30.276307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/236] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:30.310665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4923-22a8-44b6-a139-c2a7fa0fe45f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:30.325457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c142a29-99a9-46b7-8bdb-caeb859fde75 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.325781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c142a29-99a9-46b7-8bdb-caeb859fde75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.326022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c142a29-99a9-46b7-8bdb-caeb859fde75 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.327689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/237] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:30.333895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.334235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.334517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.372052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:30.375068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:30.476933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/238] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:30.501507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:30.544734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4923-22a8-44b6-a139-c2a7fa0fe45f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:30.555699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:30.603706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:30.603894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:30.604802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.604802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.605098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.627809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:30.634148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:30.635049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:30.635858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:30.641521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/239] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:30.655368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:30.670943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fea63f-a35a-4fc7-9f09-4b08605e2ae9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:30.745887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:30.746520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.746753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.749714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.752458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:30.758146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:30.760108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/240] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:30.767701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:30.842291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66517-16a1-482c-95f4-3e2574114943 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:30.883194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:30.884430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:30.884967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:30.885480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:30.886451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:30.886979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:30.888766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:30.901821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/241] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:30.907699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:30.963718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:30.976792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fae71-d387-4548-bc09-420eb30f9bf0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:31.030658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e3133-a77b-4ee3-b722-cc10b91a8e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:31.031805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3133-a77b-4ee3-b722-cc10b91a8e2c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.032657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3133-a77b-4ee3-b722-cc10b91a8e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.033213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3133-a77b-4ee3-b722-cc10b91a8e2c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.034193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e3133-a77b-4ee3-b722-cc10b91a8e2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:31.036663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e3133-a77b-4ee3-b722-cc10b91a8e2c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:31.042392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364adeb-67bb-4311-8de7-30419448c5d3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:31.049784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:31.071522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/242] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:30 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:31.073343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd0c4d-38f2-4360-aff9-4cb856eea0f1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.073865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd0c4d-38f2-4360-aff9-4cb856eea0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.074372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd0c4d-38f2-4360-aff9-4cb856eea0f1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.077752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/243] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:31.089377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.090805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.093048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.112465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:31.114194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.118999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.119590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] PATH_INFO: `/v3/users/23721eb3a4b84e3184a984ce0d0cd024` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.133790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:31.136452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:31.152116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] RBAC: Authorizing `identity:get_user(user_id=23721eb3a4b84e3184a984ce0d0cd024)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:31.154382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:31.156540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/244] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:30 2022] GET /identity/v3/users/23721eb3a4b84e3184a984ce0d0cd024 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:31.162651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:31.288350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/245] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:31.311504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:31.319763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:31.340097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4464a-6bb8-4a48-b36f-6a55be38de9c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:31.385180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:31.385998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.386424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.386746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.465021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1555e1-5a35-4f8d-bb90-c408e2de7506 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:31.477205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:31.478786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:31.500069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:31.517860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:31.518533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.518841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.519125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.532841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:31.543951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:31.544898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:31.545686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:31.546280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/246] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:31.557239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:31.593356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:31.612268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/247] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:31.620459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:31.719491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cc4e4-682c-4506-9d80-2e1855a6f5dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:31.756121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7eb47-6f5c-448a-b731-edbf14ee17bc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:31.766429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:31.767084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.767692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.768415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:31.768749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:31.771116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:31.781561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/248] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:31.789967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:31.815159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:31.817526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:31.817526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:31.817526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:31.819004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:31.820544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:31.822480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/249] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:31 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:31.829646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:31.887527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:31.934901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:32.001104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f4601-ab2d-47be-9af4-bc5826d5f1a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:32.002437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fa865-12bb-43f7-8669-8d8b409161f9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:32.066190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:32.066754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.067043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.067321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.068155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] RBAC: Adding query filter params (role.id=902e6af348874520b982b3f959494f04, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=23721eb3a4b84e3184a984ce0d0cd024) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:32.068471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:32.069948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:32.076207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/250] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:31 2022] GET /identity/v3/role_assignments?user.id=23721eb3a4b84e3184a984ce0d0cd024&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=902e6af348874520b982b3f959494f04 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:32.077745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755fcf6-f238-482a-bb8a-faa0385124fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:32.078428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755fcf6-f238-482a-bb8a-faa0385124fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.078719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755fcf6-f238-482a-bb8a-faa0385124fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.079003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755fcf6-f238-482a-bb8a-faa0385124fa None admin] PATH_INFO: `/v3/users/adb56f48867f40cbb4aa45dc38449ef2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.082853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755fcf6-f238-482a-bb8a-faa0385124fa None admin] RBAC: Authorizing `identity:get_user(user_id=adb56f48867f40cbb4aa45dc38449ef2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:32.084404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755fcf6-f238-482a-bb8a-faa0385124fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:32.086371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/251] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:31 2022] GET /identity/v3/users/adb56f48867f40cbb4aa45dc38449ef2 => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:32.092270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:32.108645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b17cf-0118-44d1-b612-08b03025bb9e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.108935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b17cf-0118-44d1-b612-08b03025bb9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.109378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b17cf-0118-44d1-b612-08b03025bb9e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.112448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/252] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:32.117515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f9ba5-4bae-4062-bde8-e7aa5c4e11f9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.118152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f9ba5-4bae-4062-bde8-e7aa5c4e11f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.118575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f9ba5-4bae-4062-bde8-e7aa5c4e11f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.139124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e67f9ba5-4bae-4062-bde8-e7aa5c4e11f9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:32.140885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e67f9ba5-4bae-4062-bde8-e7aa5c4e11f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:32.224256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fdc21-7217-453d-b012-5b9ea4f9ed08 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:32.265681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:32.272617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:32.273400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.273689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.273967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.276283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:32.278228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:32.282856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/253] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:32 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:32.288133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:32.297016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/254] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:32.314777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac405c2-29f0-4a13-baa9-9002690d8868 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.315480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac405c2-29f0-4a13-baa9-9002690d8868 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.316288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac405c2-29f0-4a13-baa9-9002690d8868 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.318417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/255] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:06:32.324843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.325425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.325913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.376553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:32.379704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:32.408642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8237-2bbe-45e3-8248-058ade18197e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:32.448510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:32.449201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.449596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.449886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252/users/adb56f48867f40cbb4aa45dc38449ef2/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.452102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] RBAC: Authorizing `identity:create_grant(project_id=faf7faecfc1b448388b564fbe426b252, user_id=adb56f48867f40cbb4aa45dc38449ef2, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:32.453746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:32.471734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/256] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:32 2022] PUT /identity/v3/projects/faf7faecfc1b448388b564fbe426b252/users/adb56f48867f40cbb4aa45dc38449ef2/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:06:32.472782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:32.474481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:32.538265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/257] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:32.555761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:32.586147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:32.654800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8363c-b0c9-4b21-8e9d-9801fd01de45 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:32.725512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:32.726189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.726473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.726753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.752848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:32.765556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:32.767401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:32.769052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:32.773923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/258] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:32.784151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:32.821691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdac8cf-622f-4cc2-a2fa-f1a13762c5c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:32.904305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:32.906676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:32.907167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:32.908628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:32.930584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:32.941955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:32.942918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:32.943794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:32.950499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/259] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:32.955476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:32.993793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc913a-8c9f-4c91-9679-9bb83415d7c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:33.008792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:33.028405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:33.129544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:33.130579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:33.131061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:33.131644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:33.132581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:33.133102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:33.134765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:33.148857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc11c5-8f2f-4544-93f1-56f6a41881ff None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:33.157683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/260] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:33.164456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:33.230387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:33.231042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:33.231333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:33.231646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:33.232378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:33.232694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:33.234456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:33.250760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/261] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:33.257431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:33.437238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff1358-ac1f-4281-93ac-8ec6cd7ebc2c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:33.503715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:33.516422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a9cea-36eb-4bdd-a042-229c3dcf52a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:33.540094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:33.540765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:33.541049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:33.541328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] PATH_INFO: `/v3/users/969b5d44f7e149db82bd279593ff8129` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:33.542660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] RBAC: Authorizing `identity:get_user(user_id=969b5d44f7e149db82bd279593ff8129)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:33.544154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:33.545807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/262] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:33 2022] GET /identity/v3/users/969b5d44f7e149db82bd279593ff8129 => generated 283 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:33.553735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:33.611424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:33.612305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:33.613066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:33.613353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] PATH_INFO: `/v3/users/969b5d44f7e149db82bd279593ff8129` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:33.615616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] RBAC: Authorizing `identity:get_user(user_id=969b5d44f7e149db82bd279593ff8129)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:33.618259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:33.619968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/263] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:33 2022] GET /identity/v3/users/969b5d44f7e149db82bd279593ff8129 => generated 283 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:33.626340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:33.635740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:33.716103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d9f70-d6f4-4587-87c7-b8cd50b1a02b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:33.765937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:33.766587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:33.766885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:33.767168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:33.796658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174cccf-227f-4800-9503-09e5962ffeb1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:33.835635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:33.836407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:33.836634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:33.836917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:33.882164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:33.883715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:33.887745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:33.921371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:33.939942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:33.942291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:34.066695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/264] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:34.072870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:34.083532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/265] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:34.108439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:34.195719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:34.224870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02dff9-7834-4958-b65b-42452684675a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:34.281117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:34.281812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.282101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.282384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.283893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:34.285974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:34.288167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/266] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:34 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:34.296514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:34.397993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99da5d1-80ff-4112-984a-d0cd8908c6a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:34.452482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:34.453147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.459511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.459511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.459511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:34.464728 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:34.466523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/267] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:34 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:34.476378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:34.488275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:34.494879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dbb65-4170-4d06-934a-61a7ae92b630 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:34.527993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:34.544251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:34.545303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.545812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.546298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.547326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] RBAC: Adding query filter params (role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=969b5d44f7e149db82bd279593ff8129) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:34.547890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:34.549573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f3c73-a8d4-4629-9f9d-4cd423f1b719 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:34.560713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/268] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:34 2022] GET /identity/v3/role_assignments?user.id=969b5d44f7e149db82bd279593ff8129&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:34.562301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33315e4b-3f73-4ffe-b44c-18472f7c4cff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.562792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33315e4b-3f73-4ffe-b44c-18472f7c4cff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.563291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33315e4b-3f73-4ffe-b44c-18472f7c4cff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.565038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/269] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:34.570195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.570774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.571287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.613271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:34.614526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:34.616588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de10557-3516-4b3d-9b02-7a80d635f8c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:34.658337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:34.658908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.659193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.660460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.661290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=969b5d44f7e149db82bd279593ff8129) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:34.661614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:34.663146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318adcfb-961d-4a03-911c-64454238f91c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:34.670266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/270] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:34 2022] GET /identity/v3/role_assignments?user.id=969b5d44f7e149db82bd279593ff8129&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:34.671671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18ef50-d02c-4f66-9905-2d158a8a24f9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.671968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18ef50-d02c-4f66-9905-2d158a8a24f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.672245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18ef50-d02c-4f66-9905-2d158a8a24f9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.673805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/271] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:34.680856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:34.681179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:34.681429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:34.702422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:34.703699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:34.781423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:34.790324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/272] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:34.791907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/273] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:34.816460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:34.836372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:34.973733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fe3f8-4ed7-4ae5-987d-4e25e404ab6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:34.998848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d5485-f6d1-4530-a341-4f8cd672732e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.005374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:35.018049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.019974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.020654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.021248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.023247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:35.025441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:35.029009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/274] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:35.034325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:35.041093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.041750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.042036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.042312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.055733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:35.060841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:35.061719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:35.062579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/275] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:35.071295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:35.150691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:35.186221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c537ac-5dab-4892-9df9-878fef8bdf93 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.199514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd9f90-17ab-4ebd-a6b9-f8fa851fbc86 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.229785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.230351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.230647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.230928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.231707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:35.233407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:35.242166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.243184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.243738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.244217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.245186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:35.245696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:35.247436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f9e1b-9dfc-4e44-8eb0-8d3a72bcb84c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:35.257157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/276] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:06:35 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:06:35.262853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/277] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:35.264278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:35.330869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63080bdc-f515-46f2-adb7-dc19547412a3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.331135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63080bdc-f515-46f2-adb7-dc19547412a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.332200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63080bdc-f515-46f2-adb7-dc19547412a3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.333786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/278] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:06:35.338259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:35.340031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.340504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.341150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.370697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:35.372281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:35.429955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1cec1-a650-4384-8b31-8c03605b597a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.460763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/279] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:35.479137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.479911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.480142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.480425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] PATH_INFO: `/v3/users/2ec87d30d2bd4107a3f25b334a99977e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.483795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:35.499167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec87d30d2bd4107a3f25b334a99977e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:35.500830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:35.502769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/280] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:35 2022] GET /identity/v3/users/2ec87d30d2bd4107a3f25b334a99977e => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:35.508730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:35.541779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:35.630699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b238ad7-eeea-4f6a-8f4e-243fdf7c78a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.652240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4a2fa-c336-40e5-833d-3a43200afdc7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.666710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:35.687376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.688090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.688385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.688690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.701849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:35.702688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.703531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.703957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.704395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.705326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, user.id=2ec87d30d2bd4107a3f25b334a99977e, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:35.705777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:35.707986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352da18-d549-4956-9c8f-ab70dae3517b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:35.708624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:35.709668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:35.710732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/281] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:35.717840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:35.718980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/282] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 05:06:35 2022] GET /identity/v3/role_assignments?user.id=2ec87d30d2bd4107a3f25b334a99977e&scope.system=all&role.id=f5c39945c9694134955532988101506f => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:35.853347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf664-db2b-4b05-bf4f-fcde33bfe2bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.853546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf664-db2b-4b05-bf4f-fcde33bfe2bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.853862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf664-db2b-4b05-bf4f-fcde33bfe2bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.854774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020194-c332-4365-be0d-1cd2146936cc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:35.855622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/283] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:35.868620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:35.876279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.876566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.876884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.901272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:35.902417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:35.902895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:35.903341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:35.904256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:35.905465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:35.906765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:35.917527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/284] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:35.920506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:35.921762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:35.923268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:36.069231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/285] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:36.107551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:36.111398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9feb12-be2e-4ef4-a53d-e105d84ec506 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:36.156811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:36.158338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:36.159076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:36.160337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] PATH_INFO: `/v3/users/23721eb3a4b84e3184a984ce0d0cd024` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:36.166279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] RBAC: Authorizing `identity:get_user(user_id=23721eb3a4b84e3184a984ce0d0cd024)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:36.169301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:36.174765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/286] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:35 2022] GET /identity/v3/users/23721eb3a4b84e3184a984ce0d0cd024 => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:36.184436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:36.282085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7153-470c-4746-9524-a380d2a891bc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:36.293405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:36.309928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:36.345256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:36.347409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:36.347850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:36.348196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:36.367204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:36.375724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:36.376707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:36.377489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:36.378230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/287] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:36.385426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:36.448741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2091fc-25a0-47f9-9511-858de27632e6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:36.541809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:36.542779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:36.543261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:36.543767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:36.545551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:36.547087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:36.552636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/288] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:36 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:36.557599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:36.662452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a994ae4-70d3-4926-b0da-05d8733f9ddc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:36.754296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:36.755326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:36.755786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:36.756208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:36.757346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:36.758391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:36.759864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:36.779819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/289] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:36.781806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:36.812563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79bc18-49c1-44b2-90cd-441ae21eae4a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:36.841067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:36.849778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:36.899265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:36.899996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:36.900282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:36.900655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252/users/23721eb3a4b84e3184a984ce0d0cd024/roles/902e6af348874520b982b3f959494f04` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:36.903588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] RBAC: Authorizing `identity:create_grant(project_id=faf7faecfc1b448388b564fbe426b252, user_id=23721eb3a4b84e3184a984ce0d0cd024, role_id=902e6af348874520b982b3f959494f04)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:36.906394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:36.925979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/290] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:06:36 2022] PUT /identity/v3/projects/faf7faecfc1b448388b564fbe426b252/users/23721eb3a4b84e3184a984ce0d0cd024/roles/902e6af348874520b982b3f959494f04 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:36.989232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c12713-6fb5-4912-8747-406362c3596f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:37.035093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:37.035912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:37.036245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:37.036532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] PATH_INFO: `/v3/users/adb56f48867f40cbb4aa45dc38449ef2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:37.037980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] RBAC: Authorizing `identity:get_user(user_id=adb56f48867f40cbb4aa45dc38449ef2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:37.039577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:37.044996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/291] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:36 2022] GET /identity/v3/users/adb56f48867f40cbb4aa45dc38449ef2 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:37.048025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:37.210715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a36b-1abd-4195-bcc4-ad7915a5cb3c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:37.253021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:37.254569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:37.255112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:37.255635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:37.309611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:37.357952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:37.359491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:37.549008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/292] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:37.556749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:37.746341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:37.767568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d61614-3d1c-4b23-bf79-cb033020f52e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:37.858028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc8f49-8e3c-4add-9b17-d706747242a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:37.859080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc8f49-8e3c-4add-9b17-d706747242a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:37.859620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc8f49-8e3c-4add-9b17-d706747242a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:37.860101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc8f49-8e3c-4add-9b17-d706747242a5 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:37.862181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc8f49-8e3c-4add-9b17-d706747242a5 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:37.865027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc8f49-8e3c-4add-9b17-d706747242a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:37.868619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/293] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:37 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:37.874365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:37.995482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:37.997224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607623e6-a2b4-4cf5-bdde-c41149e057eb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:37.997716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607623e6-a2b4-4cf5-bdde-c41149e057eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:37.998212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607623e6-a2b4-4cf5-bdde-c41149e057eb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.000939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/294] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:38.019091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a265ad3-4a23-419d-a86c-9010216b966e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.019647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a265ad3-4a23-419d-a86c-9010216b966e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.020200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a265ad3-4a23-419d-a86c-9010216b966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.039107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea77b4-de41-49a0-896c-20498e5b6711 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:38.070992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a265ad3-4a23-419d-a86c-9010216b966e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:38.073073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a265ad3-4a23-419d-a86c-9010216b966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:38.107180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:38.108613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.109066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.109558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.111105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=adb56f48867f40cbb4aa45dc38449ef2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:38.111714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:38.115246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:38.142078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/295] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:37 2022] GET /identity/v3/role_assignments?user.id=adb56f48867f40cbb4aa45dc38449ef2&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=f5c39945c9694134955532988101506f => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:38.229321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/296] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:38.268286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:38.269386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f082c3-4f0b-404c-8e0d-24b8b6bfd80f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.269875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f082c3-4f0b-404c-8e0d-24b8b6bfd80f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.270350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f082c3-4f0b-404c-8e0d-24b8b6bfd80f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.272391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/297] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:38.278443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137ccdd-1dfb-45a7-8e85-2a3795a6abd2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.279074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137ccdd-1dfb-45a7-8e85-2a3795a6abd2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.279584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137ccdd-1dfb-45a7-8e85-2a3795a6abd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.298982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1137ccdd-1dfb-45a7-8e85-2a3795a6abd2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:38.300880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1137ccdd-1dfb-45a7-8e85-2a3795a6abd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:38.387496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/298] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:38.389307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b56da-69d6-4252-a251-731418e3e55c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.389799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b56da-69d6-4252-a251-731418e3e55c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.390274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b56da-69d6-4252-a251-731418e3e55c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.391833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/299] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:38.399071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.399661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.400121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.420994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:38.421477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93566e-9f36-4d3e-a97b-96557e6898ed None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:38.422694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:38.443784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:38.464081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:38.470464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:38.471944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.474003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.474496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.475940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:38.478400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:38.480152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/300] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:38.481027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:38.516491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/301] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:38.517518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:38.602837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f8706-d54d-42f8-a5da-ac29212e341d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:38.625299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3cd37-49de-45f4-92f7-852e198c073a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:38.647814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:38.648724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.649139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.649546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.664604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:38.665553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.666050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.666230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:38.666677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] PATH_INFO: `/v3/groups` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.667562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:38.669265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:38.676264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:38.677516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:38.678483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/302] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:38.683391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:38.701803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/303] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Jul 29 05:06:38 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:38.702932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:38.731066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:38.801604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:38.872542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca880-cfee-42e4-8877-ae54c25dec04 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:38.912948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:38.913499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.913844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.914122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.919672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581876f-1404-4500-9c77-89fe9698a1ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:38.930057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:38.935790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:38.936821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:38.938005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/304] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:38.943793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:38.988915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:38.989493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:38.989777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:38.990055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:38.990792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:38.991102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:38.993300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:39.003958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/305] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:39.010547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:39.080011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3553bc6-2732-42d5-ac4c-76dd1665f85f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:39.109822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:39.132667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:39.133310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.133597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.133887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.134619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:39.134759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:39.135074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:39.136790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:39.153284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/306] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:39.159813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:39.191571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf42c-b5bd-4705-a97c-7a80cd8cb8dc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:39.251777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:39.253525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.254074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.254502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] PATH_INFO: `/v3/users/2ec87d30d2bd4107a3f25b334a99977e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.256796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec87d30d2bd4107a3f25b334a99977e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:39.258794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:39.261437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/307] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:39 2022] GET /identity/v3/users/2ec87d30d2bd4107a3f25b334a99977e => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:39.271780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:39.316507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bfd1a-549b-4988-b035-d7c04f0bd4b5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:39.360831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:39.361613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.362087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.362403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] PATH_INFO: `/v3/users/86946416cbfe4763b8f035db03a56194` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.378277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] RBAC: Authorizing `identity:get_user(user_id=86946416cbfe4763b8f035db03a56194)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:39.380018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:39.386449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/308] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:39 2022] GET /identity/v3/users/86946416cbfe4763b8f035db03a56194 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:39.387795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:39.428060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0a719-43c5-498f-aa16-c77da8fb65b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:39.452201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:39.478985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28f955-3286-4b85-bc2e-cccff23d690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:39.479922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28f955-3286-4b85-bc2e-cccff23d690e None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.480324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28f955-3286-4b85-bc2e-cccff23d690e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.480671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28f955-3286-4b85-bc2e-cccff23d690e None admin] PATH_INFO: `/v3/system/users/2ec87d30d2bd4107a3f25b334a99977e/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.482460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28f955-3286-4b85-bc2e-cccff23d690e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ec87d30d2bd4107a3f25b334a99977e, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:39.484084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28f955-3286-4b85-bc2e-cccff23d690e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:39.504064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/309] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Jul 29 05:06:39 2022] PUT /identity/v3/system/users/2ec87d30d2bd4107a3f25b334a99977e/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:39.536397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec585916-e153-47e0-a5b1-95f341887a3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:39.579093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:39.580017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.580448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.580869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.582554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] RBAC: Adding query filter params (role.id=902e6af348874520b982b3f959494f04, user.id=86946416cbfe4763b8f035db03a56194, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:39.583024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:39.584691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b97ca-506c-4b57-8383-6391d10a626c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:39.591144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/310] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 05:06:39 2022] GET /identity/v3/role_assignments?user.id=86946416cbfe4763b8f035db03a56194&scope.system=all&role.id=902e6af348874520b982b3f959494f04 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:39.725744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c69c79-be9a-4cb6-a5c1-d5b97eca2453 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.726266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c69c79-be9a-4cb6-a5c1-d5b97eca2453 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.726266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c69c79-be9a-4cb6-a5c1-d5b97eca2453 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.728151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/311] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:39.734107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:39.734601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:39.734702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:39.758865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:39.761030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:39.867662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/312] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:39.890494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c69c79-be9a-4cb6-a5c1-d5b97eca2453 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:39.894830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:40.059364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c69c79-be9a-4cb6-a5c1-d5b97eca2453 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:40.123957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:40.125131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:40.125652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:40.126158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:40.145861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:40.152120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:40.153012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:40.153778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:40.156642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/313] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:40.163524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:40.203932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:40.279914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd25548-1b2f-4899-aebe-c5574c2e16d4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:40.317766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:40.318367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:40.318670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:40.318957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:40.319725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:40.320037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:40.321463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:40.335728 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/314] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:40.337715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:40.428271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:40.475443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f3874-d637-475e-a364-dad948967148 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:40.521350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:40.522281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:40.522658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:40.523087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] PATH_INFO: `/v3/users/23721eb3a4b84e3184a984ce0d0cd024` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:40.524849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] RBAC: Authorizing `identity:get_user(user_id=23721eb3a4b84e3184a984ce0d0cd024)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:40.526850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:40.529525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/315] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:40 2022] GET /identity/v3/users/23721eb3a4b84e3184a984ce0d0cd024 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:40.538205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:40.648155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:40.655566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a448e-550d-4266-be0d-e616aaf510f9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:40.696982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0f91e-72cb-41d0-8b4a-34d16cae8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:40.697929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0f91e-72cb-41d0-8b4a-34d16cae8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:40.698427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0f91e-72cb-41d0-8b4a-34d16cae8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:40.698886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0f91e-72cb-41d0-8b4a-34d16cae8dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:40.774456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0f91e-72cb-41d0-8b4a-34d16cae8dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:40.776754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0f91e-72cb-41d0-8b4a-34d16cae8dc0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:40.887942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:40.894536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/316] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 05:06:40.900505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:41.013309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41953138-e8be-4c12-9521-41e84c0d007b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.013912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41953138-e8be-4c12-9521-41e84c0d007b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.014400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41953138-e8be-4c12-9521-41e84c0d007b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.015751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123bd9-df1d-4639-b416-303b41c32210 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:41.016555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/317] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:41.025278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.025920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.026408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.048073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:41.049652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:41.051391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:41.057279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:41.058350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.059014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.059380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.060641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:41.062491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:41.069518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/318] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:40 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:41.070744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:41.154721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/319] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:41.177715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:41.249249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc9bb-ec04-49bc-9c23-448bb608774d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:41.316910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c65cb2-3aa4-420f-9c22-90882883f61b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:41.324954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:41.325603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.325892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.326182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.326984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] RBAC: Adding query filter params (role.id=902e6af348874520b982b3f959494f04, scope.project.id=faf7faecfc1b448388b564fbe426b252, user.id=23721eb3a4b84e3184a984ce0d0cd024) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:41.327303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:41.329462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:41.338280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/320] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:06:41 2022] GET /identity/v3/role_assignments?user.id=23721eb3a4b84e3184a984ce0d0cd024&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=902e6af348874520b982b3f959494f04 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:41.355837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:41.368201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3fcd7-0f7a-417f-9f2b-69e585f86d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:41.369511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3fcd7-0f7a-417f-9f2b-69e585f86d67 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.370013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3fcd7-0f7a-417f-9f2b-69e585f86d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.370456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3fcd7-0f7a-417f-9f2b-69e585f86d67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.372317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3fcd7-0f7a-417f-9f2b-69e585f86d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:41.374427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3fcd7-0f7a-417f-9f2b-69e585f86d67 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:41.377281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/321] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:41.383425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:41.461807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:41.513274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7529158-4577-4368-b69a-4ab72e9a5fea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:41.553011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b339f9-dc64-4081-a1c8-41928eb2c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:41.553656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b339f9-dc64-4081-a1c8-41928eb2c978 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.553977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b339f9-dc64-4081-a1c8-41928eb2c978 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.554224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b339f9-dc64-4081-a1c8-41928eb2c978 None admin] PATH_INFO: `/v3/groups` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.555562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b339f9-dc64-4081-a1c8-41928eb2c978 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:41.557059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b339f9-dc64-4081-a1c8-41928eb2c978 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:41.576286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/322] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Jul 29 05:06:41 2022] POST /identity/v3/groups => generated 226 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:06:41.760448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d199-3878-49f2-b19e-48c753930e7d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.760625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d199-3878-49f2-b19e-48c753930e7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.760909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d199-3878-49f2-b19e-48c753930e7d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.762506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/323] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:41.769628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da57ea-57b6-4cf8-9998-65cd5e79bc43 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:41.769948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da57ea-57b6-4cf8-9998-65cd5e79bc43 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:41.770441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da57ea-57b6-4cf8-9998-65cd5e79bc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:41.805976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17da57ea-57b6-4cf8-9998-65cd5e79bc43 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:41.807271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17da57ea-57b6-4cf8-9998-65cd5e79bc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:41.892309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/324] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:41.916621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6d199-3878-49f2-b19e-48c753930e7d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.087328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f32b3e-6b6c-4a63-8f4e-d42031356cde None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.087836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f32b3e-6b6c-4a63-8f4e-d42031356cde None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.087836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f32b3e-6b6c-4a63-8f4e-d42031356cde None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.089235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/325] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:42.095706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.095706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.095706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.103591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6d199-3878-49f2-b19e-48c753930e7d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.125092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:42.126594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:42.161137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:42.162192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.162192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.162350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.179397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:42.193133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:42.194108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:42.196037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/326] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:42.202617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.211815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:42.243226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/327] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:42.275637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.305625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:42.388847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2efbe-694c-4357-8ecb-9fe2e17ccb9a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.439418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e0494-a0b2-4f10-aefe-56c1c323b8cf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.455502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:42.457152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.457992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.458582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.460174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:42.460804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:42.463276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:42.481598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/328] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:42.486552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:42.488195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.488343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.488672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.489217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.511752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:42.518635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:42.519975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:42.520761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/329] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:42.530130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.621505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:42.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15507ee6-201d-4930-9c47-8388aeb06b22 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.687090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48a431-5089-4f49-8651-2f4f8856f306 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.690706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:42.712705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:42.713374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.713665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.713946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] PATH_INFO: `/v3/users/2ec87d30d2bd4107a3f25b334a99977e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.715165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] RBAC: Authorizing `identity:get_user(user_id=2ec87d30d2bd4107a3f25b334a99977e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:42.716964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:42.718959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/330] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:42 2022] GET /identity/v3/users/2ec87d30d2bd4107a3f25b334a99977e => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:42.725149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.729699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:42.730345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.730631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.730911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.731685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:42.731995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:42.733411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:42.744174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/331] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:42.750154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:42.854692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88ba6-50ff-4429-aa4b-32bb25916a8b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.900013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:42.901639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:42.901927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:42.902210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:42.903016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] RBAC: Adding query filter params (role.id=f5c39945c9694134955532988101506f, user.id=2ec87d30d2bd4107a3f25b334a99977e, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:42.903333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:42.904819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:42.921894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/332] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 05:06:42 2022] GET /identity/v3/role_assignments?user.id=2ec87d30d2bd4107a3f25b334a99977e&scope.system=all&role.id=f5c39945c9694134955532988101506f => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:42.959304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c119e-f328-40d7-a2f2-1cf9b0e98cb6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:42.967771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:43.008616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506cf4c-2bfc-42b1-accd-4fb535a828cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:43.009299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506cf4c-2bfc-42b1-accd-4fb535a828cb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:43.009587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506cf4c-2bfc-42b1-accd-4fb535a828cb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:43.009870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506cf4c-2bfc-42b1-accd-4fb535a828cb None admin] PATH_INFO: `/v3/users/86946416cbfe4763b8f035db03a56194` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:43.011135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506cf4c-2bfc-42b1-accd-4fb535a828cb None admin] RBAC: Authorizing `identity:get_user(user_id=86946416cbfe4763b8f035db03a56194)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:43.012773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506cf4c-2bfc-42b1-accd-4fb535a828cb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:43.014809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/333] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:42 2022] GET /identity/v3/users/86946416cbfe4763b8f035db03a56194 => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:43.021243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:43.085872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:43.164539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49569-b52b-40f8-99c5-60eb19e8cefe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:43.206575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12379-3bf0-4908-bc58-f8e25be6ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:43.208024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12379-3bf0-4908-bc58-f8e25be6ea92 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:43.208324 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12379-3bf0-4908-bc58-f8e25be6ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:43.208602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12379-3bf0-4908-bc58-f8e25be6ea92 None admin] PATH_INFO: `/v3/system/users/86946416cbfe4763b8f035db03a56194/roles/902e6af348874520b982b3f959494f04` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:43.212092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf12379-3bf0-4908-bc58-f8e25be6ea92 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=86946416cbfe4763b8f035db03a56194, role_id=902e6af348874520b982b3f959494f04)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:43.213544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf12379-3bf0-4908-bc58-f8e25be6ea92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:43.231433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/334] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Jul 29 05:06:43 2022] PUT /identity/v3/system/users/86946416cbfe4763b8f035db03a56194/roles/902e6af348874520b982b3f959494f04 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:06:44.862660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eda49-cd2f-437e-a8bc-3c2b143edf9f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:44.863235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eda49-cd2f-437e-a8bc-3c2b143edf9f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:44.863235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eda49-cd2f-437e-a8bc-3c2b143edf9f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:44.864909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/335] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:44.882870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:44.883439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:44.883439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:44.917063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:44.918853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:45.009110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/336] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:45.056560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eda49-cd2f-437e-a8bc-3c2b143edf9f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:45.188451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eda49-cd2f-437e-a8bc-3c2b143edf9f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:45.251787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc638a4-7edc-4178-bca4-5b9fc6fd3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:45.252457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc638a4-7edc-4178-bca4-5b9fc6fd3a6a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.253457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc638a4-7edc-4178-bca4-5b9fc6fd3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.253906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc638a4-7edc-4178-bca4-5b9fc6fd3a6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.276415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc638a4-7edc-4178-bca4-5b9fc6fd3a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ccc638a4-7edc-4178-bca4-5b9fc6fd3a6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:45.282107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:45.283039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:45.283942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:45.286175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/337] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:45.291872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:45.350735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:45.443019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57fba7-594d-43e5-b3f9-cf259e5c948d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:45.463615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee42456-8af0-458a-84af-26bb1bc11a2e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.463949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee42456-8af0-458a-84af-26bb1bc11a2e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.464187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee42456-8af0-458a-84af-26bb1bc11a2e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.465829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/338] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:45.470648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.470944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.471222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.497362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:45.498390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.498900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.499431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.500436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:45.501325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:45.503044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:45.506631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:45.508291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:45.522120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/339] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:45.528722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:45.562886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:45.650450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/340] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 29 05:06:45.687504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:45.715144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:45.717868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c4122-c9c3-48b2-85f2-8759ec932b15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:45.766570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0c935-8668-4815-8125-87d09b6158ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:45.767489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c935-8668-4815-8125-87d09b6158ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.768012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c935-8668-4815-8125-87d09b6158ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.768471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c935-8668-4815-8125-87d09b6158ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.857024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0c935-8668-4815-8125-87d09b6158ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:45.860530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0c935-8668-4815-8125-87d09b6158ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:45.948430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bbb32-65c4-42eb-9785-6f28642e8404 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:45.983937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:45.989549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/341] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:45.991696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c1b1-3200-475b-ab58-508e1dbd91e3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.992172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c1b1-3200-475b-ab58-508e1dbd91e3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.992769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c1b1-3200-475b-ab58-508e1dbd91e3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.994316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/342] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:45.995785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382c8df-e8ca-4da2-b9e1-1f1449eafce4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:45.996250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382c8df-e8ca-4da2-b9e1-1f1449eafce4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:45.996905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382c8df-e8ca-4da2-b9e1-1f1449eafce4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:45.998602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/343] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:46.005181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7b78f-9179-4d07-8482-34d829972209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.005694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.006145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.006267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7b78f-9179-4d07-8482-34d829972209 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.006473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7b78f-9179-4d07-8482-34d829972209 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.006759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7b78f-9179-4d07-8482-34d829972209 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.006984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.021758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7b78f-9179-4d07-8482-34d829972209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-12c7b78f-9179-4d07-8482-34d829972209 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:46.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:46.028217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:46.029174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/344] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:46.030546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.030844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.031127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.033747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:46.035771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:46.053283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:46.055006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:46.079092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:46.125454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/345] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:46.126604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.149924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/346] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:46.150898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.258318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6303faf-3134-4642-aea4-3d042c23b205 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.286490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76311b7-9c0a-4fcb-bcff-8a8208fa56b6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.304758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:46.317626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.319059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.319715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.320369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.322448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:46.324100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.324828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.325193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.325550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.325969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:46.326431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:46.326819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:46.328337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:46.329480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/347] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:46.331783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.340021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/348] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:46.341080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.361944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:46.452856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb437-5f53-439e-a4a0-5c386fca287b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.463668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5b7f7-63de-4f2b-ae28-9e07ce35dc79 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.500620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.502084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.503411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.504133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.505015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.505752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.506108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.506451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.518806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:46.524640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:46.526266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/349] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:46.527894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.531154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:46.546018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:46.548077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:46.550305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:46.554995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/350] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:46.556786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.601425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:46.671797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b7b4c-e560-42d3-8e5f-220268345529 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.722218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.723067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.723441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.723807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.726557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:46.733880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:46.735903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:46.738367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/351] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:46.739306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:46.777892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf1bd0-7735-4f80-8013-1db75407696a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.837017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.837017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.837305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.837476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.896419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced432e-5437-45cd-bbe5-8fe4bd64de27 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:46.921334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:46.933801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:46.935418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:46.944170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:46.945434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:46.945724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:46.946006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:46.946751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:46.948435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:46.948885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:46.963300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/352] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:46.964288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.047314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:47.086087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/353] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:47.086087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.105886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9183799-cfcc-4b47-9cf1-593959ec3e6d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:47.151754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:47.152401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.152684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.152966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.153693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:47.154007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:47.155472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2bd3-24f7-473c-9a05-15b7dc1be48b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:47.165488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/354] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:47.166779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39b73-a7b0-49db-8697-2f03834cd4a9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.167067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39b73-a7b0-49db-8697-2f03834cd4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.167435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39b73-a7b0-49db-8697-2f03834cd4a9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.168710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/355] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:47.169444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39b73-a7b0-49db-8697-2f03834cd4a9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.228656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6114-bae3-4bc7-98a2-530f565557a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:47.239245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:47.273035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:47.273680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.273963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.274243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.275043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=f5c39945c9694134955532988101506f, scope.project.id=d5c486928a4e4e68b08704afe7cd789d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:47.275492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:47.276903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:47.282792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/356] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:47 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=f5c39945c9694134955532988101506f => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:47.283695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.333371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39b73-a7b0-49db-8697-2f03834cd4a9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:47.342772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:47.387340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da2cf5-e4be-49ab-8544-53a8cfa9a8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:47.388051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da2cf5-e4be-49ab-8544-53a8cfa9a8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.388343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da2cf5-e4be-49ab-8544-53a8cfa9a8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.388627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da2cf5-e4be-49ab-8544-53a8cfa9a8a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.429011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b413a-a7db-4e4d-97a2-ca32ccf4359c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:47.473770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d27f6-986e-4843-a8c8-07341a58527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:47.474369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da2cf5-e4be-49ab-8544-53a8cfa9a8a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:47.474543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d27f6-986e-4843-a8c8-07341a58527a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.474727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d27f6-986e-4843-a8c8-07341a58527a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.474999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d27f6-986e-4843-a8c8-07341a58527a None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.475923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da2cf5-e4be-49ab-8544-53a8cfa9a8a8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:47.477462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d27f6-986e-4843-a8c8-07341a58527a None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:47.478950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d27f6-986e-4843-a8c8-07341a58527a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:47.480750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/357] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:47 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:47.481828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310de8aa-d4f4-46c2-b804-57c4b8620682 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.482118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310de8aa-d4f4-46c2-b804-57c4b8620682 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.482400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310de8aa-d4f4-46c2-b804-57c4b8620682 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.491389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/358] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:47.491389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310de8aa-d4f4-46c2-b804-57c4b8620682 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.547571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:47.637522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/359] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:47.639442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1443b-c845-45b2-a69f-3b9ff02ca7e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.639735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1443b-c845-45b2-a69f-3b9ff02ca7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.640016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1443b-c845-45b2-a69f-3b9ff02ca7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.653703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310de8aa-d4f4-46c2-b804-57c4b8620682 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:47.663910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-11c1443b-c845-45b2-a69f-3b9ff02ca7e9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:47.664675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:47.665134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-11c1443b-c845-45b2-a69f-3b9ff02ca7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:47.699799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:47.700979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.701477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.701961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.761742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/360] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:47.763499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:47.764024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:47.764303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:47.784695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:47.786083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:47.808150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:47.810027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:47.884751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/361] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:47.886498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.890288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:47.965772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/362] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:47.967174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:47.990746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.053724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332bb16-b599-4527-84ea-54251f7b11ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.107411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.108357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.108814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.109287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.112832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.115515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.117461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/363] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:47 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.118349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.129995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ca160-42be-4348-aab4-ea33c0a4dd31 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.180755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.181205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.181485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.181760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.182989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.184768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.186615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/364] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:47 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.187432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.207448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.256980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05db7-e995-429d-bde7-c1ba769794a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.291493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.312979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.313808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.314092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.314717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.323395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca5ee9-dd8a-4ba2-8b5a-8643237bdd18 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.333434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:48.340016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:48.341051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:48.341867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/365] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:48.342530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.378415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.379540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.380039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.380514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.402117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:48.407922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:48.408932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:48.410224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/366] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:48.411013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.490237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b71f7-f15a-4dd8-8cbb-5a135a357d59 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.527159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.536412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.536844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.537130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.537411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.538858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.540508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.542448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/367] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.543090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.559755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20838-af89-4e77-9201-1451cf019083 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.617384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.618030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.618322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.618603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.619502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=d5c486928a4e4e68b08704afe7cd789d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:48.619817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.621292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.624927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.639383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/368] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.639383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.681525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a844b-221c-4f90-a802-d59e6fb3c92a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.730451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.731126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.731455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.731737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.733051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.734539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.736761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/369] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.743322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.765960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1e35-2896-4e0b-a00a-cdf8045893f8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.806370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.807205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.808454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.808790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.809566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:48.809951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.811385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.822121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/370] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.822852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.849887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.896005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:48.902674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14aac8-8806-41e7-9f8a-fa376759bc7d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.944011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.944649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.944934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.945211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.945934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:48.946246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.946517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab59aab-c37e-4f0d-aa8d-578003a75afd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:48.947663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.957800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/371] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.959018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:48.990481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:48.991896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:48.992468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:48.992939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:48.994483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:48.996297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:48.998609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/372] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:48.999657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:49.083463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c37b98-ca78-4953-b353-166a290cca82 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:49.116714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:49.123157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:49.123832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.124563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.124953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.126192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=faf7faecfc1b448388b564fbe426b252) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:49.126862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:49.127766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c6df-5a67-4db8-b8a0-4ea6e37d97ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:49.129013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:49.135705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/373] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:49.140548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:49.162428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:49.169652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:49.170781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.171221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.172034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] PATH_INFO: `/v3/users/86946416cbfe4763b8f035db03a56194` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.173860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] RBAC: Authorizing `identity:get_user(user_id=86946416cbfe4763b8f035db03a56194)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:49.175681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:49.178313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/374] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:06:48 2022] GET /identity/v3/users/86946416cbfe4763b8f035db03a56194 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:49.179456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:49.284535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca797e53-54bd-426b-b7c4-5816bc0341ea None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:49.328701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3053a-d0ad-4671-b3b4-f3d43b7a868b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:49.341405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d39ea7-6b17-4423-9471-0ea8cca407c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:49.342065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39ea7-6b17-4423-9471-0ea8cca407c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.342350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39ea7-6b17-4423-9471-0ea8cca407c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.342688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39ea7-6b17-4423-9471-0ea8cca407c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.416526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:49.417451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.417972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.418487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.420157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=f5c39945c9694134955532988101506f, scope.project.id=faf7faecfc1b448388b564fbe426b252) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:49.423735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:49.426059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:49.441966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/375] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:49 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=f5c39945c9694134955532988101506f => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:49.444011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:49.473015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:49.482384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d39ea7-6b17-4423-9471-0ea8cca407c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:49.484163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d39ea7-6b17-4423-9471-0ea8cca407c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:49.591821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:49.606570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/376] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:49.609305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3300-6d50-4b3f-9ca7-9c90b5ce801b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.609897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3300-6d50-4b3f-9ca7-9c90b5ce801b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.610424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3300-6d50-4b3f-9ca7-9c90b5ce801b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.614625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/377] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:49.616187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d3300-6d50-4b3f-9ca7-9c90b5ce801b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:49.667742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37784bf4-59c9-483a-8297-ac3f8c248798 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:49.728396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:49.729176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.729461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.729740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.730687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] RBAC: Adding query filter params (role.id=902e6af348874520b982b3f959494f04, user.id=86946416cbfe4763b8f035db03a56194, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:49.731010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:49.732646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103010d-4ebe-46f1-8c3e-c84e7d7421c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:49.747421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/378] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 05:06:49 2022] GET /identity/v3/role_assignments?user.id=86946416cbfe4763b8f035db03a56194&scope.system=all&role.id=902e6af348874520b982b3f959494f04 => generated 510 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:49.748722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.749169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:49.749739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.750106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.761314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d3300-6d50-4b3f-9ca7-9c90b5ce801b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:49.772351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:49.773774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:49.821024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b148528-7005-4836-9861-021c76206a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:49.821741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b148528-7005-4836-9861-021c76206a3a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:49.822028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b148528-7005-4836-9861-021c76206a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:49.822305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b148528-7005-4836-9861-021c76206a3a None admin] PATH_INFO: `/v3/projects/3340905a228b4864923e8ce0f5cfd8a4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:49.824608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b148528-7005-4836-9861-021c76206a3a None admin] RBAC: Authorizing `identity:get_project(project_id=3340905a228b4864923e8ce0f5cfd8a4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:49.826153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b148528-7005-4836-9861-021c76206a3a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:49.830995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/379] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:49 2022] GET /identity/v3/projects/3340905a228b4864923e8ce0f5cfd8a4 => generated 344 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:49.837612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b148528-7005-4836-9861-021c76206a3a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:49.971138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/380] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:49.996062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:50.063023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b148528-7005-4836-9861-021c76206a3a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:50.087113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:50.122545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:50.124107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:50.124544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:50.124968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] PATH_INFO: `/v3/groups/9fa90c486c8b420c9b2facf8f1825b48` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:50.137717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] RBAC: Authorizing `identity:get_group(group_id=9fa90c486c8b420c9b2facf8f1825b48)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:50.139479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:50.142210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/381] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:49 2022] GET /identity/v3/groups/9fa90c486c8b420c9b2facf8f1825b48 => generated 229 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:50.149076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:50.188352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:50.201200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6e50d-b1ab-4ffe-a4ab-8257a5e0bf03 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:50.279011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:50.279011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:50.279477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:50.279883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:50.307524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4894b74-640c-4587-9d1a-f067a5151606 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:50.308547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:50.313919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:50.315036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:50.315990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:50.322635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/382] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:50.326752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:50.375546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:50.376178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:50.376463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:50.376740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:50.377577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] RBAC: Adding query filter params (group.id=9fa90c486c8b420c9b2facf8f1825b48, role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=3340905a228b4864923e8ce0f5cfd8a4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:50.378006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:50.379549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:50.391500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/383] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:50 2022] GET /identity/v3/role_assignments?group.id=9fa90c486c8b420c9b2facf8f1825b48&scope.project.id=3340905a228b4864923e8ce0f5cfd8a4&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:50.485730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16166650-7e45-4a1b-a85f-349ce95d937c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:50.497566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:50.533622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:50.534215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:50.534504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:50.534775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:50.535528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:50.535835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:50.537245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:50.547922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/384] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:50.553804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:50.686929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:50.702724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae8b38-867d-459d-b824-0509bafec487 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:50.749541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:50.750598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:50.751101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:50.751697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:50.754071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:50.755869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:50.758085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/385] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:50 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:50.763281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:50.886851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d643a6-6a0d-47fe-9069-c3be42fea932 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:50.931283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809a835-a8ac-4dca-98ff-457a617cff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:50.932521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a835-a8ac-4dca-98ff-457a617cff18 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:50.933047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a835-a8ac-4dca-98ff-457a617cff18 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:50.933632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a835-a8ac-4dca-98ff-457a617cff18 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:50.935388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809a835-a8ac-4dca-98ff-457a617cff18 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:50.937318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809a835-a8ac-4dca-98ff-457a617cff18 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:50.939847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/386] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:50 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:50.946871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:51.098598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61011f5-4b93-4d8d-81d0-29f55ea9ed3a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:51.152375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9ebad-1ea8-43ac-92c6-9c58e12f066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:51.154781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9ebad-1ea8-43ac-92c6-9c58e12f066f None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:51.155292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9ebad-1ea8-43ac-92c6-9c58e12f066f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:51.155848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9ebad-1ea8-43ac-92c6-9c58e12f066f None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:51.161334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9ebad-1ea8-43ac-92c6-9c58e12f066f None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c486928a4e4e68b08704afe7cd789d, group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:51.163186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9ebad-1ea8-43ac-92c6-9c58e12f066f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:51.182476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/387] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 05:06:50 2022] PUT /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:51.201010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4560353-0239-4b2b-85ed-13d75fb6aa7e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:51.201306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4560353-0239-4b2b-85ed-13d75fb6aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:51.202020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4560353-0239-4b2b-85ed-13d75fb6aa7e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:51.203289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/388] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:51.210919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:51.211705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:51.212084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:51.255100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:51.256617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:51.281691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:51.385699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/389] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:51.422754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4560353-0239-4b2b-85ed-13d75fb6aa7e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:51.479401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:51.576215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4560353-0239-4b2b-85ed-13d75fb6aa7e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:51.631591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efca267-a377-41cf-bdb0-8d1b925fa28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:51.632259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efca267-a377-41cf-bdb0-8d1b925fa28c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:51.632546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efca267-a377-41cf-bdb0-8d1b925fa28c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:51.632826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efca267-a377-41cf-bdb0-8d1b925fa28c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:51.653770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efca267-a377-41cf-bdb0-8d1b925fa28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7efca267-a377-41cf-bdb0-8d1b925fa28c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:51.659731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:51.661027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:51.662215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/390] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:51.670391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:51.727658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1771a9-962a-4ecf-807e-1edbfb25ea02 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:51.727856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1771a9-962a-4ecf-807e-1edbfb25ea02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:51.728137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1771a9-962a-4ecf-807e-1edbfb25ea02 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:51.729857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/391] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:51.735925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:51.736267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:51.736543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:51.785800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:51.790611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:51.947085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5edd0c-45f7-4756-9d63-e5746ea1e8ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:51.966272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/392] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:52.003646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.004242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.004579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.004831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.005556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:52.005865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:52.007294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:52.023494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/393] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:52.028884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.030213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.050467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:52.090157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:52.235692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d28f5-8d34-415e-aa96-09252b0885ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.246451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b6402-00bf-4260-b98b-3ec98a90fa6c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.284371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.285145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.285430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.285710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.308347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:52.321090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:52.322072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:52.322858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/394] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:52.328905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.329656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.329969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.330392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.332173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:52.333759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:52.335723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/395] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:52.341544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.343420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.474066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:52.509718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c528bf-201e-4858-b25f-4546f9497372 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.511779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:52.523656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a380-4777-42a5-adb8-b152bac5e892 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.553590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96867a00-c0c0-4989-b31f-45c223716a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.554439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96867a00-c0c0-4989-b31f-45c223716a58 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.554853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96867a00-c0c0-4989-b31f-45c223716a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.555314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96867a00-c0c0-4989-b31f-45c223716a58 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.556854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96867a00-c0c0-4989-b31f-45c223716a58 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:52.558616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96867a00-c0c0-4989-b31f-45c223716a58 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:52.560537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/396] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:52.561985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cab3f-a98e-4a40-9011-32dda1bc85e5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.562454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cab3f-a98e-4a40-9011-32dda1bc85e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.562812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cab3f-a98e-4a40-9011-32dda1bc85e5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.567839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.568489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.568900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/397] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:52.569071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.569473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.569973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cab3f-a98e-4a40-9011-32dda1bc85e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.570253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:52.570691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:52.572284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211740-533f-4beb-88e3-3363556a1f8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:52.587483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/398] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:52.589263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.589558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.589837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.612372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:52.613680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:52.690761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cab3f-a98e-4a40-9011-32dda1bc85e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.698732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/399] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:52.699637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.756623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.757642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.758073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.758491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.762648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c486928a4e4e68b08704afe7cd789d, group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:52.765429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:52.788245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/400] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 05:06:52 2022] PUT /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:52.789305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.800396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:52.833158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:52.857341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8e2a7-f421-41f5-9adc-0348f45b1da8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.908854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.909824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.910369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.910873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:52.912999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:52.916441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:52.920463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/401] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:52.927997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:52.930568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0f607-e8fd-4dc7-82d0-2dfa8a01b847 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:52.992432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:52.993449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:52.993949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:52.994423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.012744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:53.022345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:53.024643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:53.026695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/402] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:53.031934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:53.128892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73bf4d-648a-4bb5-8731-8c7f7ad63921 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:53.164225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:53.190814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:53.191713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.191998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.192334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.194021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:53.197987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:53.198622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/403] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:52 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:53.205626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:53.235410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:53.240625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d448073-1d81-475b-87cc-ccae0af11509 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:53.302529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:53.303983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.304497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.304977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.305939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:53.306475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:53.308121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:53.324845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/404] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:53.335300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:53.429101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c304aa-d30b-417f-aa8d-a4c9ed7cd528 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:53.496022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:53.496935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.497230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.497537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.501965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=faf7faecfc1b448388b564fbe426b252, group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:53.506156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:53.534395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/405] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 05:06:53 2022] PUT /identity/v3/projects/faf7faecfc1b448388b564fbe426b252/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:06:53.568930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad2cda-9f4a-4e45-a59a-82312ac55e70 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:53.615779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:53.626218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfac2b3-3f04-4843-857a-9423e7e099b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:53.626952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfac2b3-3f04-4843-857a-9423e7e099b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.627853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfac2b3-3f04-4843-857a-9423e7e099b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.628399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfac2b3-3f04-4843-857a-9423e7e099b7 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.629780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfac2b3-3f04-4843-857a-9423e7e099b7 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:53.631448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfac2b3-3f04-4843-857a-9423e7e099b7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:53.633691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/406] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:53 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:53.644939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:53.666949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18b08-1136-4677-82ac-bcf9b03f9209 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.667280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18b08-1136-4677-82ac-bcf9b03f9209 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.667719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18b08-1136-4677-82ac-bcf9b03f9209 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.669800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/407] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:53.677751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.678151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.678325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.724679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:53.725828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:53.754077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:53.774008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d48b49-d2b8-4b9a-af53-b476ff8f76b3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:53.836266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006983-3680-4903-b920-47806584f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:53.841391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006983-3680-4903-b920-47806584f3be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:53.841721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006983-3680-4903-b920-47806584f3be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:53.841958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006983-3680-4903-b920-47806584f3be None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:53.848217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46006983-3680-4903-b920-47806584f3be None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:53.849707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46006983-3680-4903-b920-47806584f3be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:53.851801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/408] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:53 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:53.860371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006983-3680-4903-b920-47806584f3be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:53.874971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/409] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:53.910704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:53.979464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006983-3680-4903-b920-47806584f3be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:54.014753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb78780-d0c0-4c92-a693-5fe1866f25b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:54.015492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb78780-d0c0-4c92-a693-5fe1866f25b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.015779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb78780-d0c0-4c92-a693-5fe1866f25b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.016055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb78780-d0c0-4c92-a693-5fe1866f25b6 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.018256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb78780-d0c0-4c92-a693-5fe1866f25b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=faf7faecfc1b448388b564fbe426b252, group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:54.022902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb78780-d0c0-4c92-a693-5fe1866f25b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:54.046972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/410] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 05:06:53 2022] PUT /identity/v3/projects/faf7faecfc1b448388b564fbe426b252/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:06:54.051449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163eddd3-41da-41eb-854c-0004a45f63f9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.051449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163eddd3-41da-41eb-854c-0004a45f63f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.051449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163eddd3-41da-41eb-854c-0004a45f63f9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.051449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/411] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:54.069463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.069463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.069463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.098341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:54.114185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d739961-8b51-495a-9c15-9e32046423ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:54.120698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:54.121933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:54.151647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:54.163672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:54.165383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.165944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.166466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.187740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:54.194159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:54.195199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:54.196052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:54.205605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/412] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:54.214905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:54.293688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/413] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:54.320318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:54.382537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502eba5c-923b-4e57-9918-40b6a586529d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:54.435302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:54.438062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.438449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.438735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.441129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:54.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:54.451390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:54.479925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/414] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:54.485370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:54.530644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287c777-f635-445b-8b3d-ba634fdb62df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:54.600140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:54.601768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:54.602685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.602793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.603064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.610670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:54.626771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:54.632226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:54.633106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:54.633894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:54.634971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/415] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:54.640934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:54.736090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bbe3-38d3-4a21-95b5-246a6dbead07 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:54.852808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:54.853878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:54.854386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:54.854885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] PATH_INFO: `/v3/groups/9fa90c486c8b420c9b2facf8f1825b48` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:54.857694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] RBAC: Authorizing `identity:get_group(group_id=9fa90c486c8b420c9b2facf8f1825b48)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:54.860834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:54.870752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/416] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:54 2022] GET /identity/v3/groups/9fa90c486c8b420c9b2facf8f1825b48 => generated 229 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:54.878143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:54.933170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecf7e5-2952-48b4-bdbe-2d476ab443e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:55.020298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:55.021399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:55.021924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:55.025876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:55.027322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:55.027922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:55.034631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:55.067660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/417] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:55.077691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:55.142428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819169a9-6577-4dc8-8a85-7f69aab8124c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:55.178575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:55.217390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:55.218576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:55.219135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:55.219679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] PATH_INFO: `/v3/projects/3340905a228b4864923e8ce0f5cfd8a4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:55.223466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] RBAC: Authorizing `identity:get_project(project_id=3340905a228b4864923e8ce0f5cfd8a4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:55.226175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:55.229951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/418] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:54 2022] GET /identity/v3/projects/3340905a228b4864923e8ce0f5cfd8a4 => generated 344 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:06:55.242111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:55.371566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:55.371798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf42e-308e-44d6-bbe5-20c70335d9a5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:55.419706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9974a02-78c6-4ef6-865b-7b186dce33ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:55.424325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:55.425231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:55.425685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:55.425979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] PATH_INFO: `/v3/projects/3340905a228b4864923e8ce0f5cfd8a4/groups/9fa90c486c8b420c9b2facf8f1825b48/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:55.432354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] RBAC: Authorizing `identity:create_grant(project_id=3340905a228b4864923e8ce0f5cfd8a4, group_id=9fa90c486c8b420c9b2facf8f1825b48, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:55.433936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:55.469977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:55.471736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:55.472444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:55.473079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:55.473940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/419] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 05:06:55 2022] PUT /identity/v3/projects/3340905a228b4864923e8ce0f5cfd8a4/groups/9fa90c486c8b420c9b2facf8f1825b48/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:06:55.560456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:55.562923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:55.634679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:55.687004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/420] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 05:06:55.696530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:55.756139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:55.819767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bde74a-6bca-4227-b15f-1824b78dc80a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:55.859802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb1e6d-1b64-41c2-a5ec-e6fcd655c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:55.860486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb1e6d-1b64-41c2-a5ec-e6fcd655c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:55.860777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb1e6d-1b64-41c2-a5ec-e6fcd655c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:55.861058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb1e6d-1b64-41c2-a5ec-e6fcd655c5a3 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:55.862785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb1e6d-1b64-41c2-a5ec-e6fcd655c5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:55.864543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb1e6d-1b64-41c2-a5ec-e6fcd655c5a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:55.866648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/421] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:55 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:55.872161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:55.992646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4631a-ebb0-4861-a6f7-3b47db461e10 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:55.992818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4631a-ebb0-4861-a6f7-3b47db461e10 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:55.993099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4631a-ebb0-4861-a6f7-3b47db461e10 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:55.994788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/422] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:56.006334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.006636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.006912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.019513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942737d3-9aad-4946-9cb6-c7c88d51e86e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:56.063376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:56.063376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:56.066866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:56.067575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.067866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.068148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.069365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:56.070842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:56.072840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/423] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:55 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:56.081572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:56.151728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:56.201789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/424] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:56.232422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:56.281408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:56.291646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e18959-d89d-4a81-88ac-8138bcecce9c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:56.349286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b490c-ed2e-4ded-8e9f-b3de2fa8e52b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:56.360350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:56.361306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.361419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.362189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.363045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=f5c39945c9694134955532988101506f, scope.project.id=d5c486928a4e4e68b08704afe7cd789d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:56.363375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:56.364965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:56.383093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/425] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:56 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=f5c39945c9694134955532988101506f => generated 611 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:56.390170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03af3c-de5c-4f01-a4d4-5407ac18c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:56.391273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03af3c-de5c-4f01-a4d4-5407ac18c637 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.391780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03af3c-de5c-4f01-a4d4-5407ac18c637 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.392295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03af3c-de5c-4f01-a4d4-5407ac18c637 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.404996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03af3c-de5c-4f01-a4d4-5407ac18c637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ca03af3c-de5c-4f01-a4d4-5407ac18c637 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:56.411451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:56.412401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:56.413230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:56.415692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/426] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:06:56.421945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:56.534240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89ca6-e69c-4411-ace9-e228ca2595de None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.535103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89ca6-e69c-4411-ace9-e228ca2595de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.535636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89ca6-e69c-4411-ace9-e228ca2595de None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.537581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/427] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:56.543509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516bb79-5cf7-4cd3-8d92-f09198bdb870 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.544137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516bb79-5cf7-4cd3-8d92-f09198bdb870 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.544646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516bb79-5cf7-4cd3-8d92-f09198bdb870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.569330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4516bb79-5cf7-4cd3-8d92-f09198bdb870 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:56.570560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4516bb79-5cf7-4cd3-8d92-f09198bdb870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:56.584613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:56.596587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4486f2-cd17-41b2-a7e2-4fc694484723 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:56.647603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:56.648883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.649321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.649995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.651304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:56.651962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:56.654523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc1d63-e42f-4398-9182-eeae080d326b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:56.664488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/428] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:06:56.665991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa61c26-08ed-409d-8426-d4d83b95da83 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.666362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa61c26-08ed-409d-8426-d4d83b95da83 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.666643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa61c26-08ed-409d-8426-d4d83b95da83 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.669362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/429] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:56.669493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/430] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:56.673738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa61c26-08ed-409d-8426-d4d83b95da83 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:56.675602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.676103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.676658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.703058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:56.704401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:56.721318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:56.788301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/431] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:56.789467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:56.813672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa61c26-08ed-409d-8426-d4d83b95da83 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:56.867641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:56.868932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.869661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.870078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.911505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:56.935756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06b04b-8604-47a9-ac0c-f3b11775757e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:56.964261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:56.965971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:56.973717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:56.974384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:56.974672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:56.974950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:56.986229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:56.992110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:56.993167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:56.994092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/432] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:06:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:56.994881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.004459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.088837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/433] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:57.089615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.112264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d0bc5-48c6-456e-9bbc-0ccce5a05920 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.149952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.150640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.150930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.151221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.165271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:57.170652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:57.171693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 05:06:57.172527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:06:57.176088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/434] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:57.176504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.180616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.222704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341049dd-87b3-4b1e-b655-184503a26742 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.255116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.262405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.263513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.264020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.264537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.265518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:57.266042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.268662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.283889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/435] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:57.285496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.305716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed2c24-1682-4086-8d59-258ca7be7977 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.358608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.359375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.359725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.360004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] PATH_INFO: `/v3/projects/d5c486928a4e4e68b08704afe7cd789d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.362687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c486928a4e4e68b08704afe7cd789d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.363961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.370635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/436] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/projects/d5c486928a4e4e68b08704afe7cd789d => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:57.370635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.427330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177d4ef-6155-4757-9271-09904ee6f87f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.442744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.463811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.464448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.464750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.465003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.465713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:57.466023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.467430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.476912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/437] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:57.477747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.485076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a7d1-ff9c-415f-935f-0094e2545ef5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.526449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.527238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.527393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.527700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.547030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.605352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93084774-b60f-49de-a036-d04a8909a857 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.607597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.609153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.641289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.642246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.642726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.643567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.644837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.646293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.653041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/438] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:57.653041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.690535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.728646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/439] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:57.734419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.765380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4bc1a-3396-44ea-9ab3-4e1e7035346b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.796117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:57.814540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bfff2-35ef-49bf-86f1-d4607d4caa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.816739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfff2-35ef-49bf-86f1-d4607d4caa87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.817260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfff2-35ef-49bf-86f1-d4607d4caa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.817871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfff2-35ef-49bf-86f1-d4607d4caa87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.850513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799b8fc-bc21-43f0-bcbd-a08199a8de4d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:57.891490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:57.892124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:57.892475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:57.892859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:57.894794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=d5c486928a4e4e68b08704afe7cd789d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:57.895117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.896834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.903102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bfff2-35ef-49bf-86f1-d4607d4caa87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:57.904170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/440] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=d5c486928a4e4e68b08704afe7cd789d&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:57.905108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bfff2-35ef-49bf-86f1-d4607d4caa87 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:57.905467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:57.952501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.038770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df58a-cc31-47e2-ac0f-389bf43e569f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.049674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/441] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:58.049674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2d897-3a4c-47a8-9fea-59e309f44712 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.049674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2d897-3a4c-47a8-9fea-59e309f44712 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.049674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2d897-3a4c-47a8-9fea-59e309f44712 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.049674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/442] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:06:58.053537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2d897-3a4c-47a8-9fea-59e309f44712 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.083451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.098338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af272e-a4d8-4dd3-acb3-df95819dae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.098906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af272e-a4d8-4dd3-acb3-df95819dae87 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.099195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af272e-a4d8-4dd3-acb3-df95819dae87 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.099508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af272e-a4d8-4dd3-acb3-df95819dae87 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.100808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af272e-a4d8-4dd3-acb3-df95819dae87 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.102326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af272e-a4d8-4dd3-acb3-df95819dae87 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.104409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/443] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:57 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.107009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.107009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.107009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.130132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:06:58.130132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:06:58.191648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2d897-3a4c-47a8-9fea-59e309f44712 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.222893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/444] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:06:58.224107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.244536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.245329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.245621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.246208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] PATH_INFO: `/v3/projects/faf7faecfc1b448388b564fbe426b252` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.247435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] RBAC: Authorizing `identity:get_project(project_id=faf7faecfc1b448388b564fbe426b252)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.248900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.255157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/445] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/projects/faf7faecfc1b448388b564fbe426b252 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.255157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.260478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.357416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a081-9595-4a21-859e-ec5cefc7cac2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.377322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.387083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c490b4-ad98-42d1-9534-d50433b8f118 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.392328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.393512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.393819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.394111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.395341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.396845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.398642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/446] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.399243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.430149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.430823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.431109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.431421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.443510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:06:58.448896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:06:58.449837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:06:58.450623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/447] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:06:58.451284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.504147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.507063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb259-7560-4475-af95-334c87cd1200 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.544021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.544694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.544981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.545300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] PATH_INFO: `/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.546457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] RBAC: Authorizing `identity:get_group(group_id=a5a777ee1c9a47dfa5bb97723ca1d3e0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.547944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.549775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/448] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/groups/a5a777ee1c9a47dfa5bb97723ca1d3e0 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.550590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.567511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61b21f-7c03-4403-ae1a-bc5502c4c3be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.604210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.604844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.605144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.605414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.606210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=faf7faecfc1b448388b564fbe426b252) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:58.606525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.608027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.613878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/449] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.614693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.635714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.666604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2960c-2858-46c5-9088-4e2d8a3de973 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.712224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.712784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.713136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.713418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.714212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:58.714529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.715988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.731692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/450] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.733217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:58.734603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc748007-3c16-4ef7-8f6e-d656484acdbe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.771120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:58.776272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.777211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.777501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.777784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.778599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] RBAC: Adding query filter params (group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0, role.id=f5c39945c9694134955532988101506f, scope.project.id=faf7faecfc1b448388b564fbe426b252) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:58.778916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.780426 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:58.787503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/451] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/role_assignments?group.id=a5a777ee1c9a47dfa5bb97723ca1d3e0&scope.project.id=faf7faecfc1b448388b564fbe426b252&role.id=f5c39945c9694134955532988101506f => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:58.857979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6672c2-874b-46a2-938e-0126265df97b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:58.895111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:58.895783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:58.896071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:58.896346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:58.967854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:58.969257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:59.000422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:59.076600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/452] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:06:59.083622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:59.171844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:59.193166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a84bf5-36c8-4951-9a47-5bab19b4d72e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:59.233716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:59.234304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:59.234592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:59.234872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] PATH_INFO: `/v3/projects/3340905a228b4864923e8ce0f5cfd8a4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:59.236094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] RBAC: Authorizing `identity:get_project(project_id=3340905a228b4864923e8ce0f5cfd8a4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:59.237775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:59.239508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/453] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:06:59 2022] GET /identity/v3/projects/3340905a228b4864923e8ce0f5cfd8a4 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:59.245171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:59.366879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc2fd8-69e7-4593-81e8-1cfabe2c1cd7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:59.395568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:06:59.401460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c70fc-23b0-4069-b9ce-b86f1927cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:59.402107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c70fc-23b0-4069-b9ce-b86f1927cc8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:59.402395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c70fc-23b0-4069-b9ce-b86f1927cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:59.402675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c70fc-23b0-4069-b9ce-b86f1927cc8c None admin] PATH_INFO: `/v3/groups/9fa90c486c8b420c9b2facf8f1825b48` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:59.403837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c70fc-23b0-4069-b9ce-b86f1927cc8c None admin] RBAC: Authorizing `identity:get_group(group_id=9fa90c486c8b420c9b2facf8f1825b48)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:59.405293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c70fc-23b0-4069-b9ce-b86f1927cc8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:59.406887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/454] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 05:06:59 2022] GET /identity/v3/groups/9fa90c486c8b420c9b2facf8f1825b48 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:06:59.413498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:06:59.514221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a1e58-38c6-4545-9e50-150e7be67f3d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:06:59.548644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:06:59.549739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:06:59.550030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:06:59.550313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:06:59.551098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] RBAC: Adding query filter params (group.id=9fa90c486c8b420c9b2facf8f1825b48, role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=3340905a228b4864923e8ce0f5cfd8a4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:06:59.551444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:06:59.553140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232d43c-16a9-42f7-9b63-28c7fba947ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:06:59.558881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/455] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 05:06:59 2022] GET /identity/v3/role_assignments?group.id=9fa90c486c8b420c9b2facf8f1825b48&scope.project.id=3340905a228b4864923e8ce0f5cfd8a4&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:02.198906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f29e0-6e7d-4781-8d49-683655603061 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.199434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f29e0-6e7d-4781-8d49-683655603061 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.199509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f29e0-6e7d-4781-8d49-683655603061 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.201157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/456] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:02.208025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae99276-29db-4c95-b1a1-9890aefbe1c8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.208532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae99276-29db-4c95-b1a1-9890aefbe1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.208605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae99276-29db-4c95-b1a1-9890aefbe1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.237522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dae99276-29db-4c95-b1a1-9890aefbe1c8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:02.238486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dae99276-29db-4c95-b1a1-9890aefbe1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:02.250070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:02.344421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/457] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:02.381578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f29e0-6e7d-4781-8d49-683655603061 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:02.432591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed422-616a-47b6-a5c6-191aacce436e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.432943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed422-616a-47b6-a5c6-191aacce436e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.433166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed422-616a-47b6-a5c6-191aacce436e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.434799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/458] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:02.440861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d0203-f95c-477d-acee-587ee82b8bc4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.441557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d0203-f95c-477d-acee-587ee82b8bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.442149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d0203-f95c-477d-acee-587ee82b8bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.483238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e64d0203-f95c-477d-acee-587ee82b8bc4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:02.484409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e64d0203-f95c-477d-acee-587ee82b8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:02.529627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f29e0-6e7d-4781-8d49-683655603061 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:02.562720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/459] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:02.564889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fe285-37ea-439e-a71c-460797104bc4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.565248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fe285-37ea-439e-a71c-460797104bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.565698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fe285-37ea-439e-a71c-460797104bc4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.570035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:02.570709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.571000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.571286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.571410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/460] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:02.573210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.573572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.573858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.585038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:02.586565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:02.594486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:02.595891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:02.603988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:02.605199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:02.606618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:02.606618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:02.606618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:02.606618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:02.606618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/461] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:02.606618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:02.663800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:02.679478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/462] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:02.680793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:02.684777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:02.723291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a58da-1a6b-43b3-b332-f909f9585493 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:02.762649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:02.763593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.764033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.764459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.777494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:02.779275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:02.795969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:02.797043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:02.798096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:02.798096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:02.798096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:02.798096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:02.800671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/463] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:02.801829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:02.818810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e9f2e-8aae-4df7-9783-edeaec0380b3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:02.856030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:02.856616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.856903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.857181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.857878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:02.858186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:02.859707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:02.873387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/464] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:02.874377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:02.922239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02195a-fa96-4c0e-af9b-d6543552050e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:02.925283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:02.951186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:02.967436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:02.968145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:02.968453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:02.968736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:02.981035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:02.982925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:02.995870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ea27-9896-4f05-b34c-7f348000b1fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:02.998156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:02.998991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:03.000013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:03.000013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:03.000013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:03.000013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:03.000013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/465] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:03.004287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.037100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.037655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.037943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.038285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.038990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:03.039300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.040740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cb32b-b1ac-4106-919b-737dc7c5b357 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.057015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/466] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:03.058315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7db7a-b091-4d15-844b-e40e552f6b25 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.058614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7db7a-b091-4d15-844b-e40e552f6b25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.058893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7db7a-b091-4d15-844b-e40e552f6b25 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.060329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/467] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:03.060883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7db7a-b091-4d15-844b-e40e552f6b25 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.134199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf523a9-16dd-44e3-ba59-531e23360188 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.177954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dc069-8f87-4b32-a47c-08aae6610f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.178698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dc069-8f87-4b32-a47c-08aae6610f39 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.178995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dc069-8f87-4b32-a47c-08aae6610f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.179278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dc069-8f87-4b32-a47c-08aae6610f39 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.180039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dc069-8f87-4b32-a47c-08aae6610f39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.181510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dc069-8f87-4b32-a47c-08aae6610f39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.185510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7db7a-b091-4d15-844b-e40e552f6b25 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.219590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/468] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:07:02 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:03.220593 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:03.222188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.222577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.222880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.231090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.232240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.232754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.233239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.234168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:03.234691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.237503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.246198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:03.247496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:03.250942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:03.266142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/469] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:03.268683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.362183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/470] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:03.364834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.429834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa702b0-80ec-458f-8eff-b38e92f8c873 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.483225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.484137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.484612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.485039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.485877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.488307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.514925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/471] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:07:03 2022] POST /identity/v3/users => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:03.516253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.529404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63674939-9f66-4b77-bee5-a6e290bedc41 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.575609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:03.584727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60170c10-1bea-44a2-98b6-4fb1598d4e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.585442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170c10-1bea-44a2-98b6-4fb1598d4e67 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.585972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170c10-1bea-44a2-98b6-4fb1598d4e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.586304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170c10-1bea-44a2-98b6-4fb1598d4e67 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.587632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170c10-1bea-44a2-98b6-4fb1598d4e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.589518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170c10-1bea-44a2-98b6-4fb1598d4e67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.607723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:03.628409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/472] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:07:03 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:03.631230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673a5a-bb0c-4e8d-9ab7-2af62f3524a7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.631817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673a5a-bb0c-4e8d-9ab7-2af62f3524a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.632277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673a5a-bb0c-4e8d-9ab7-2af62f3524a7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.635642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/473] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:03.642144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.643439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.644652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.681696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d7eeb-9315-4c26-8107-b8e8c19c6f2f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.684717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:03.687255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:03.743261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.745158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.745451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.745736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.760505 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.762601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:03.783769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:03.785181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:03.786022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:03.786022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:03.786022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:03.786022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:03.786022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/474] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:03.793403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.797543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/475] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:07:03.829888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:03.910568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:03.921705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe3f7-b72f-4f44-a8a9-58c634237a38 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.963015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf54ae-087f-42c1-92fc-ec5f29a1902f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:03.963664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:03.964319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:03.964614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:03.964998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:03.965699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:03.966006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:03.968664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:03.986486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/476] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:03.992262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:04.003905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:04.016380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:04.017316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:04.017753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:04.018040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:04.019087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:04.021085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:04.031960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 05:07:04.031960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 05:07:04.031960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [parameters: {'id': 'e2f64bd775c449c0b700c1e1e4ad43c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 05:07:04.031960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 29 05:07:04.061528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 05:07:04.062459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 05:07:04.064058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 05:07:04.065289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 05:07:04.066725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 05:07:04.068905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': 'e2f64bd775c449c0b700c1e1e4ad43c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:04.070772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 05:07:04.075650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:04.077693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/477] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 05:07:03 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:04.077693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:04.162655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7587e-8814-487c-8ccf-f237dcfcba76 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:04.236685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:04.238226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:04.238675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:04.239824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:04.240717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:04.242496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:04.264074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8033e8-0260-499f-a934-338c0493ce53 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:04.270302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/478] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:07:03 2022] POST /identity/v3/users => generated 249 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:04.315256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd171-88a8-453e-851b-7ece23de4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:04.316278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd171-88a8-453e-851b-7ece23de4137 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:04.316703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd171-88a8-453e-851b-7ece23de4137 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:04.317120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd171-88a8-453e-851b-7ece23de4137 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:04.329585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fd171-88a8-453e-851b-7ece23de4137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-548fd171-88a8-453e-851b-7ece23de4137 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:04.335942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:04.337106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 05:07:04.338079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:04.340683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/479] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:04.346814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:04.367751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:04.493087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2f02-26b3-4efb-924a-1f39a6892077 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:04.529588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:04.530224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:04.530508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:04.530782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:04.531535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:04.531903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:04.533360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874c7e-19fd-45d4-a646-3831fc6834a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:04.542457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/480] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:05.848316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac417cca-239e-43e8-98cb-6b528a9fd816 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:05.848842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac417cca-239e-43e8-98cb-6b528a9fd816 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:05.848950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac417cca-239e-43e8-98cb-6b528a9fd816 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:05.850606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/481] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:05.856319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6f1d-bd84-4c83-bc9f-13dc6af933cd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:05.856794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6f1d-bd84-4c83-bc9f-13dc6af933cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:05.856984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6f1d-bd84-4c83-bc9f-13dc6af933cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:05.884262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-146b6f1d-bd84-4c83-bc9f-13dc6af933cd None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:05.885490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-146b6f1d-bd84-4c83-bc9f-13dc6af933cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:05.968565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:05.977725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/482] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:07:06.011139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac417cca-239e-43e8-98cb-6b528a9fd816 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.079742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:06.143794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac417cca-239e-43e8-98cb-6b528a9fd816 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:06.151697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089082d7-4d3f-4e8f-9317-88e5e3f31d1b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.152043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089082d7-4d3f-4e8f-9317-88e5e3f31d1b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.152301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089082d7-4d3f-4e8f-9317-88e5e3f31d1b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.154006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/483] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:07:06.159745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.160038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.160324 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.186346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:06.187605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:06.197488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:06.198521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.199004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.199512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.217273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:06.223467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:06.224407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:06.225290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:06.228104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/484] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:06.233861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.317342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/485] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:07:06.342717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.460711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79182cd1-23a4-4d5c-884d-2d15ba22f438 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:06.488618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bfc10-ea73-4fac-9e4b-27bf7a710a58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:06.496598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:06.500022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:06.502562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:06.504638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.505144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.505626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.517623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:06.523164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:06.524112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:06.524916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:06.527674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/486] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:06.533919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.560130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:06.560785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.561080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.561362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.562106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:06.562472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:06.563888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24887b6d-17cf-4ffb-89c6-01d379b2d29f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:06.575159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/487] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:06.576624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645fd4-93e0-4d0b-96f7-5ceafa3da9fd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.576967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645fd4-93e0-4d0b-96f7-5ceafa3da9fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.577248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645fd4-93e0-4d0b-96f7-5ceafa3da9fd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.583306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/488] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:06.584385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645fd4-93e0-4d0b-96f7-5ceafa3da9fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.666700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3e050-cd27-4839-9f22-cb7150fa4a07 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:06.712794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:06.714398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.714928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.715442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.716381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:06.716914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:06.719361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed95fa4-1b6a-44d5-b70c-0ef6db572968 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:06.737835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645fd4-93e0-4d0b-96f7-5ceafa3da9fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:06.738052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/489] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:06.738052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.738190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.738654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.758005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:06.759329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:06.781304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:06.782768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:06.784169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:06.784690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:06.801045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:06.804103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:06.814114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:06.823982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:06.824884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:06.825736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:06.825736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:06.825736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:06.825736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:06.828108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/490] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:06.829481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.847533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/491] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:06.848583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:06.862077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:06.971377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762eedf-4d03-4c1c-8663-2da3741c84ca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:06.994475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e73a-d955-4051-a34b-f001326d9d28 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.033890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.035272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.036333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.036970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.057763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.059004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.059630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.060202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.060895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.061724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:07.062456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.064783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.065976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.089499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/492] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:07.091119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:07.093829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:07.094700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:07.095799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:07.095799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:07.095799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:07.095799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:07.095799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/493] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:07.097584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.158758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.200660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.226387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7143c8a-dcca-416a-8b7d-66a6c88c72d4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.228716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f7fd-320b-4512-95e4-801db2e93a2a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.260257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.260904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.261191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.261472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.263063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.263904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.264678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.264957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.269652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.271132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.275648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:07.280933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:07.281885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:07.282662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/494] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:07.284689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-31b8d33c-2315-47f0-9766-14b6b1593204 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:07.284797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:07.286587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/495] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c048-78cc-43fe-807c-c3b3f81d9f8a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.289150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c048-78cc-43fe-807c-c3b3f81d9f8a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.290733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c048-78cc-43fe-807c-c3b3f81d9f8a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.291750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/496] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:07.293170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549596a1-f78c-493e-8834-7a5dbde13969 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.293464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549596a1-f78c-493e-8834-7a5dbde13969 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.293744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549596a1-f78c-493e-8834-7a5dbde13969 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.295290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/497] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:07.295791 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549596a1-f78c-493e-8834-7a5dbde13969 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.390883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.395306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc6c09-31be-4919-bb21-dac50ff23e27 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.406269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549596a1-f78c-493e-8834-7a5dbde13969 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.430594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.431226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.431545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.431828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.432537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:07.432845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.434238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e514b56-c39c-4383-857d-af4f5e2128d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.446628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.447204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/498] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:07.447582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.448005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.448415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.448510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d709c3e-4953-4ef3-af98-c781220888a6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.448777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d709c3e-4953-4ef3-af98-c781220888a6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.449056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d709c3e-4953-4ef3-af98-c781220888a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.449654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:07.450104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.452395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.468243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.476306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d709c3e-4953-4ef3-af98-c781220888a6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:07.477583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d709c3e-4953-4ef3-af98-c781220888a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:07.496203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/499] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:07.496879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.548915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/500] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:07.550641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.551094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.551548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.578741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:07.580881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:07.598236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e834690-56a6-42e4-ae16-2c0939ee6f39 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.633676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.634298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.634579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.634854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.635601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:07.635909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.637308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.646222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/501] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:07.647171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.652016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/502] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:07.652893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.676379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.709083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.756988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c93be1-8ab9-465d-998f-7a6b804fef03 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.761226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95ba15-26ba-4cee-98b7-d6e683e2c89a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.792684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.793298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.793584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.793857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.799960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.800595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.800881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.801158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.802688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.804220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:07.816290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:07.817208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/503] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:07.818543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.876603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.878631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.900913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.922253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e192d45-730c-45ad-8101-82e6c09c6811 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:07.940479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:07.960067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:07.960725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:07.961018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:07.961297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:07.972766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:07.974268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:07.980451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/504] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:07.981301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:07.994028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:07.994917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:07.995874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:07.995874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:07.995874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:07.995874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:07.995874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/505] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:07.996149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.087985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ab95d-13ac-47da-8f92-6ab1e211e983 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.109328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.110818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c01f5-a3b6-4e45-82aa-7fd388709dd2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.124297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.124948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.125234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.125518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.136344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.137877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.146169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.146816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.147579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.147865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:08.155236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:08.156219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:08.157046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:08.157046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:08.157046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:08.157046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:08.157046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/506] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:08.157299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.162344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:08.167848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:08.168728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:08.169529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/507] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:08.170181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.201654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.267037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c11dc5-b11d-4ef3-98ce-ad4ca4af03f5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.277936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13d450-9645-42a6-a6f8-baecd72a8d7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.301547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.302429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.302743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.303027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.303858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:08.304175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.305621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.312299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.312954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.313243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.313524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.324743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.326235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:08.341429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:08.342282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:08.347867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:08.347867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:08.347867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:08.347867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:08.347867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/508] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:08.348217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/509] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:08.348305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.348383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.374994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.430599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.447505 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a63196-f990-4c1f-baf1-0330b36131ce None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.453094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ac6c-8186-4ce8-990a-3296870bd85f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.483014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.483682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.483972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.484255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.484952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:08.485265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.486714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.487476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.488104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.488385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.488659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.489356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:08.489657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.491016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.498334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/510] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:08.499078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.509486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/511] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:08.510330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.599694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.609298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eac42d-16c1-4593-9f23-c92825a74c8c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.611690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2101aee-df01-4d2e-8d70-0e9aa7aa01ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.643455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.644078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.644363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.644636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.645345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:08.645652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.646227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.646852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.647032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.647141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.647454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.648151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:08.648465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.649848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.655261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/512] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:08.656089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.662046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.665699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/513] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:08.666835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.760652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd23a9-bf05-4dea-a23b-10d8d40756b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.768140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e6368-48f4-4120-bd6f-db2294222914 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.800132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.801051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.801478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.802154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.805011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.805671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.805958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.806242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.816107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.817588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.831912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:08.837210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:08.838273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/514] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:08.840504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:08.895208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.896750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.906879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:08.949940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3095f8d-7ebb-4f49-bac3-61ff5d6cdd69 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:08.983895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:08.984519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:08.984803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:08.985081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:08.985768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:08.987872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:08.998632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/515] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:08.999613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.015328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/516] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:07:08 2022] POST /identity/v3/users => generated 247 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:09.016479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.060725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.123664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d3ff-9f5b-47c0-bfd1-58fafa2e398d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.149968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70327c29-cba3-4f0f-8ed5-4bde53216fe8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.164834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.165511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.165806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.166089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.170856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.181891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.183951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.195614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.197064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.198130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.199090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:09.202313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:09.203114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:09.204009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:09.204009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:09.204009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:09.204009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:09.204009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/517] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:09.204775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.212420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.214703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:09.227576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:09.229286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/518] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:09.230549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.333060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3e3e-5a6f-42f3-a3fb-909593b817a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.336881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.350189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f4e7-018f-4603-a23c-0d73197b5dee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.375684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.376319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.377133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.377410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.378340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:09.378767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.380313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.386445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.387108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.387429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.387719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.400339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.405927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/519] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:09.406079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.406170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:09.421038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:09.421947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:09.422789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:09.422789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:09.422789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:09.422789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:09.422789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/520] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:09.423047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.455169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.519715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325395d-d384-43bf-b8a1-eb6c94dfbc61 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.529579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62489aff-648b-4bb7-a96b-ea25c3843348 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.560268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.560909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.561244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.561677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.562135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:09.562458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.563893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.570540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.571181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.571988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.572275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.573033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:09.573788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.575249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.581385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/521] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:09.581385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.598462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/522] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:09.599424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.608840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.693027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e97a48-326e-4d0b-98bf-bb6c5fb7b957 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.702472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.717005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd630-a776-4603-9134-086237e6da84 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.730512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.731147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.731487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.731775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.758332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.758962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.759248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.759822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.760326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:09.760618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.762015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.778786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/523] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:09.778786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.816029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.818549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.846415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:09.898840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4e1e7-1d02-46c9-a65a-d8e8a7050c6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:09.931500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/524] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:09.931500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.941587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:09.942376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:09.942659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:09.942940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:09.951648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:09.953055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:09.966230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:09.967185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:09.968467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/525] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:09.969120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:09.983581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:10.044408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae7a70-5023-4f08-a5c2-98452c72075b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.073535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:10.090240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.090835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.091126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.091435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.092292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=7c858ed2d88a43afaf34e83155629c97) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:10.092607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.094191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.111643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/526] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/role_assignments?user.id=7c858ed2d88a43afaf34e83155629c97&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:10.111643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.111927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5636288-211d-46bf-b4fb-478e001166c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.145718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.146437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.146726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.147002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.157592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.159390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:10.174915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:10.175863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:10.177041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:10.179516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/527] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:10.180550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.258465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5a140-5903-457a-b023-dcefd35d9258 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.297161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:10.310034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.310808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.311202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.311671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.328825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.332101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.337163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe76ba0-9777-4fd6-a5f0-dbcb69a070e0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:10.352060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:10.353047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:10.353872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:10.353872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:10.353872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:10.353872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:10.355433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/528] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:10.356383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.383313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.384036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.384348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.384756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.385573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:10.385891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.387388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.406620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/529] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:10.407588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.407803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:10.503979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70e166-2ffb-4b76-a8c5-c40834330050 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.529532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54d9e-97b8-4a40-a21d-31192b2f3afb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.555260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.555931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.556245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.556614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.557537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:10.557855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.559449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.560693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:10.570145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.570779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.571068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.571386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.572177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:10.572493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.573900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.582777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/530] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:10.583976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.590999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/531] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:10.591971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.695940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:10.700538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae787f-1cb5-42b6-97f8-22ba1f86c744 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.718533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c4ccb-99ed-4893-984e-5cb01b52dd25 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:10.742719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.743844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.744875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.745399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.762288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:10.762842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:10.763131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:10.763440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:10.764547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1d28b235f4434aa086cdfc0fc81c7551) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:10.764867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.766618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.776479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/532] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/role_assignments?user.id=1d28b235f4434aa086cdfc0fc81c7551&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:10.777599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:10.862314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:10.865161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:10.875398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.004043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4cee7-0d50-4dcb-ae4e-0f223ced134e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.029784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.036239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/533] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:11.041769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.073936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.074942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.075382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.075783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.087904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:11.089440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:11.107363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:11.108283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:11.109128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/534] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:11.115466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.235663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f50238-50d7-4d63-874b-f2df1334e3fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.245650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51a738-b979-44e3-9429-c58c1e674cfa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.261172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.281492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.282180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.282360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.282638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.293534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.294039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:11.294238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.294444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.294721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.295506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:11.295683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:11.295835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:11.297737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d0c2b861-1a31-453b-8051-fdf2bbe3a0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:11.312760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:11.315401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/536] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5124463-1cc6-4261-8312-70409aa87f6c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5124463-1cc6-4261-8312-70409aa87f6c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5124463-1cc6-4261-8312-70409aa87f6c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.318580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/537] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:11.326598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.326885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/537] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:11.326885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5124463-1cc6-4261-8312-70409aa87f6c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.380096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.437881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5124463-1cc6-4261-8312-70409aa87f6c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.439834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714284-aa46-4bf4-afd5-1c1e2c6f35b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.475427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.476285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.476665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.477039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.477842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.478031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=0a47f6ceb2a146b092ebeb72e8679335) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:11.478445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:11.478558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.478741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.479021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.479737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:11.480045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:11.480471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8edd06-f5a7-4032-a3af-540e84fd3085 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:11.481479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:11.488452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/538] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/role_assignments?user.id=0a47f6ceb2a146b092ebeb72e8679335&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:11.489886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.490277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.490696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.506357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/539] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:11.512105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.516802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:11.518743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:11.584038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.682154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab338e-16a5-46b0-8749-8008c9033cea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.690875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/540] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:11.723597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.724603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.725356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.725637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.725966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.741608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:11.744722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:11.767316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:11.768250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:11.769107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:11.774182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/541] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:11.782041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:11.811721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.894450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df73c5-1a55-46e0-b465-477fa3c33100 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.959334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:11.965406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:11.966064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:11.966344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:11.966618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:11.979424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea3019-9ad2-4bf6-976f-29b3d03da4a6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:11.993187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:11.998558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:11.999496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:12.000319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:12.003271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/542] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:12.009126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:12.053272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:12.054642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.055120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.056726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:12.057281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:12.059078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:12.083942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/543] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:12.090275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:12.161383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b87c8-5918-4417-a05d-2b1742abdafd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:12.167661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:12.222992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb29b47-393a-4d14-94fe-606b8f92f29a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:12.256601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:12.257587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.258113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.258583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.259526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:12.260028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:12.261623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:12.277363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/544] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:12.281579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:12.282742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:12.284574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.285462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.288210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.289238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=eac8ab3df29343c5bfa8928000f99b28) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:12.289753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:12.291433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a16ac-7484-4308-9f94-218fc771f878 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:12.306129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/545] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:12 2022] GET /identity/v3/role_assignments?user.id=eac8ab3df29343c5bfa8928000f99b28&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:12.425235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1584940-d0fc-43b0-a258-1118aab1dc6e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:12.433440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117f37-7d8a-4139-90fe-125130850525 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.433751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117f37-7d8a-4139-90fe-125130850525 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.434003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117f37-7d8a-4139-90fe-125130850525 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.435518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/546] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:07:12.453342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.453674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.453900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.468263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:12.469280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.469786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.470367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.482960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:12.484892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:12.502814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:12.504554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:12.520353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:12.521349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:12.522178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:12.522178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:12.522178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:12.522178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:12.522178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/547] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:12.526788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:12.528830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:12.585342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:12.620519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/548] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:07:12.662451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:12.746294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2b50-da53-472d-817b-62e78876caf7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:12.817038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:12.818412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:12.819747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:12.820230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:12.822992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:12.823567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:12.826816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:12.859451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/549] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:12.868614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:12.907074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc3525-0e3e-45a1-a6d4-7d2a1a5450a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.000706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.001285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.002251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.002850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.022754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:13.026041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:13.034245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:13.035955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:13.037121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/550] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:13.042136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.053386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392f4ca-deb6-4a38-8bfa-235b2cf37a60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.058654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.093747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.094415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.094703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.094987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.105491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.107114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7e87ab97-3068-46a6-944c-28f3d1dc24aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:13.122441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:13.123477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:13.124559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/551] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:13.125648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa06d5-5071-4c9c-9e41-5413605247a1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.125846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa06d5-5071-4c9c-9e41-5413605247a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.126126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa06d5-5071-4c9c-9e41-5413605247a1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.132563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/552] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:13.133547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa06d5-5071-4c9c-9e41-5413605247a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.193705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241148-baf0-493e-bca2-dae69550c055 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.242762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.243701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.243999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.244286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.245034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:13.245411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.247697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0400e5-0329-4a6d-b9be-a88c15221e7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.259174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/553] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:13.260647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.260948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.261231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.271063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa06d5-5071-4c9c-9e41-5413605247a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.281710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:13.283673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:13.315868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.317349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.317926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.318462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.319673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:13.320219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.321886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.333560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.346871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/554] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:13.348298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.354358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.371852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/555] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:13.372658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.464835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1970048-dc1f-460a-82ab-075bd177892e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.502387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec17a91-bfb4-43d8-b494-075e566fd3a3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.506003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.506930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.507446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.507943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.521364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.523674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.541331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.541981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.542265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.542547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:13.550014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:13.551571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:13.553208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:13.553208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:13.553208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:13.553208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:13.556694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/556] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:13.558482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.616602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.617430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.628112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.630539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.696028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b49d3-2d30-48c6-ba95-a4f39b8bb6d3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.736865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.737940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.738425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.738920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.750555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:13.753427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/557] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:13.754436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:13.757657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:13.759026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:13.759882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:13.762648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/558] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:13.763772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.866754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.876009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfaa87-faac-47d0-b6d4-e5808403b08c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.878528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344b888-ea6c-40ec-b22e-fdf21f285fbd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:13.888852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:13.917165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.918233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:13.918500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.918866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.919154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.919436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.919538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.919993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.920178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:13.920461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.922054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.934349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:13.935648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/559] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:13.935897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:13.936702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0c799df4-c2ed-4e4f-b745-89b78e4a6acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:13.953448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:13.954511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:13.956519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:13.956519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:13.956519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:13.956519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:13.956519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/560] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:13.956519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ed416-520a-4899-8f9b-830272a1484a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:13.956857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ed416-520a-4899-8f9b-830272a1484a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:13.956926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ed416-520a-4899-8f9b-830272a1484a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:13.962466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/561] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:13.962466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ed416-520a-4899-8f9b-830272a1484a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.054549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013f0f-e162-4c31-be1e-10625e3a1f81 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.080696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ed416-520a-4899-8f9b-830272a1484a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.091624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.092378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.092666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.092952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.093673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:14.093978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.096105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.109101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/562] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:14.109101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.122438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.123026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.123319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.123626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.127229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.131319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.132833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-202628ec-5a79-438a-8933-345ffdcce7ff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:14.147975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:14.149251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:14.150187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/563] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:14.150795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.151082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.151392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.170047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:14.171281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:14.187119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.233504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42b4b3-9494-4160-b888-44cb4c3138d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.247333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/564] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:14.248369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.278140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.279122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.279649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.280620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.281527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:14.282063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.284229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.296349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/565] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:14.297281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.374584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5e45-7c41-4e69-ac52-49963c0be4a2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.388421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.413423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133eae6-7e4a-4f40-9f7a-2c21fedfaacb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.415298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.415991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.416274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.416551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.428159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.429639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:14.445714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:14.446685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:14.447712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:14.447712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:14.447712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:14.447712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:14.447712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/566] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:14.447712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.453699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.454832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.455614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.455914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.456545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.457356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:14.457699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.459274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.477800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/567] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:14.478609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.577861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06757af-5ce8-40e2-811d-b1cd3c3e8e5a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.585493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8e9ea-dda1-4060-a99c-07f245b89274 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.618164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.618765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.619047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.619319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.621832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.622772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.623061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.623360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.627019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.631249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.631636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:14.633143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:14.637687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:14.638610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:14.639492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:14.643739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/568] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:14.644421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:14.650841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:14.651768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/569] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:14.653410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.712389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.753957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da530595-9f0a-4744-ba72-e7f368f4dd13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.758017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a373b3a-84df-4902-a9ce-d7818e807d3f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.788710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.789341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.789631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.789915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.790609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:14.790920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.792337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.792468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.792986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.793274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.793556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.804942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/570] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:14.805598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.806130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.807681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c62af7fb-a1bc-4534-8b31-4acb15fdf4ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:14.822283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:14.823469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:14.824411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:14.824411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:14.824411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:14.824411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:14.827502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/571] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:14.828994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8fbb1-723c-4c5e-8baf-5cee3ee478da None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.829293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8fbb1-723c-4c5e-8baf-5cee3ee478da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.830076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8fbb1-723c-4c5e-8baf-5cee3ee478da None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.832159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/572] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:14.832873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8fbb1-723c-4c5e-8baf-5cee3ee478da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.886324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.912438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fef93-0e3b-470f-aa04-ce9dc6f73367 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.936301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:14.942803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8fbb1-723c-4c5e-8baf-5cee3ee478da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:14.949019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.949653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.950369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.950656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.951406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:14.951726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.953802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.962782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/573] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:14.963636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:14.977738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:14.978368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:14.978652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:14.978929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:14.979732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:14.980044 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:14.981492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:14.994211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/574] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:14.995031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.068565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e5fa-9c25-4281-b34e-e9a68d1f05de None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.095521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dba3c-1d21-45b7-9f11-db31a5df30e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.104202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.104838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.104950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.105137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.105418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.114405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.115911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a0d38aa6-20c6-462c-be5a-95ed8d411e7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:15.127791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:15.128870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:15.129879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/575] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:15.131060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.131060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.131060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.132494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.133131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.133647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.133974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.134687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:15.134994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.136503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.149888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/576] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:15.150800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:15.151044 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.152133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:15.200674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.228496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/577] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:15.230509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.258089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0648e9b-ab3c-400b-80e8-23ae40fd4916 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.291872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.292515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.292798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.293070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.304633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.306104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:15.320847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:15.321925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:15.324392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:15.324392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:15.324392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:15.324392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:15.324392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/578] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:15.324392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.340571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d8e9-ef85-4d03-aa19-1ff6245cd490 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.348935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.375809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.376961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.377298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.377589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.378388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1176e3df3023409b93375dc94ffb661c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:15.378706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.380190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.385662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/579] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/role_assignments?user.id=1176e3df3023409b93375dc94ffb661c&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:15.386449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.437493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.442690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017c839-af42-40e0-9be9-d7e5d532c878 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.489579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.490247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.490531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.490803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.491770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:15.492083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.493564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.512105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2d2b4-8d44-4282-9fba-aace07e6b435 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.513569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/580] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:15.516348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.563373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.563373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.563373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.563373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.568045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.569717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:15.582771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:15.583727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/581] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:15.589810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.606637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.639235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e17f74-ecb7-4f31-87f6-6f2b8f47e37d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.689974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.690735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.691141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.691573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.703181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:15.710090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:15.711057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:15.711944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/582] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:15.712659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.724334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.729411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b5a17-db90-4ec5-b3d6-fd346d0ad768 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.767816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.768959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.769243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.769523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.770236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:15.770532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.772131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.785532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/583] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:15.786225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.832151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21f267-94ef-4023-a8a5-6b15c9528912 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.869486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.870295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.870686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.871091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.878503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:15.883537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.885116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:15.900660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:15.901774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:15.902869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:15.902869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:15.902869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:15.902869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:15.909234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/584] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:15.909234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.917529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0756-bf69-45b8-a198-8e77b02623f0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:15.955368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:15.955966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:15.956221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:15.956497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:15.957260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:15.957572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:15.959086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:15.977422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/585] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:15.978199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:15.999807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.019180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b91ef-0494-4bc9-8ebc-4f4b47532381 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.056106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.056735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.057031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.057315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.058050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:16.058364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.059849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.069732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/586] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:16.070621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.100459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92788a43-f468-4d3a-bf62-dd8724e440f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.135623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.141088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.141747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.142089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.142373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.150443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.151962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:16.163508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:16.164476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/587] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:16.172305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.191302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8aeb90-53e2-411f-af7c-1b421d47983c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.228826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.229380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.229668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.229948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.230639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:16.230951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.232364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.245390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/588] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:16.246193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.268239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.292760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44071d62-50c5-4068-85c2-6286ee84dc70 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.333479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.334159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.334477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.334804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/7c858ed2d88a43afaf34e83155629c97/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.352401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=7c858ed2d88a43afaf34e83155629c97, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.354019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.354820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ccca5-27f1-4c60-9162-3f0a4f7ba9d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.370553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.379472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/589] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:16 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/7c858ed2d88a43afaf34e83155629c97/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:16.382029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.398789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.399612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.399888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.400168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.414132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.415709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:16.434778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:16.435882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:16.436777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:16.436777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:16.436777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:16.436777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:16.441965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/590] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:16.443655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.526690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae24efe-6f0e-4cbd-8148-eff4bd2f5732 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.565421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f00d2-c428-49d6-99a8-bbc63d40bfd8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.566059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.580811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.581453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.581741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.582019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.582819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:16.583131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.585661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.604532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/591] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:16.605468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.616052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.616852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.617669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.618057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.628580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.630262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:16.646968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:16.648144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:16.649276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:16.651531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/592] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:16.652806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.685722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.735239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb910dd-86e1-4e6f-8ad8-ae8b15220630 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.775516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad004a-8b03-4ee4-b5c9-8d276a038412 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.784462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.785880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.786389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.787304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.788519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:16.789037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.791440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.809250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/593] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:16.810350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.817747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.818527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.818878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.819230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.834169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.836708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:16.857570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:16.858541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:16.859510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:16.859510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:16.859510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:16.859510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:16.865487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/594] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:16.865487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:16.872709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.943854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ae98-d745-4171-868e-7635de528242 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:16.963268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:16.984491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:16.985185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:16.985522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:16.986404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:16.987264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:16.987607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:16.989057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:16.993089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9889ba-9a50-4355-a0f4-56c8a453c5d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.003411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/595] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:17.004354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.042423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.043011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.043298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.043762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.055158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.056759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:17.071854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:17.072869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 05:07:17.073825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:17.079837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/596] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:17.079837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.142393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60575f-c7eb-4f69-8db0-c25353b9a19a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.154456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:17.193975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.194679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.194967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.195252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.198299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:17.198299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.198496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.212572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249c6e5-6627-4bbe-b7cd-135b1827ef29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.214704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/597] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:17.216446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.262526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.263901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.265297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.265798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/1d28b235f4434aa086cdfc0fc81c7551/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.279743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:17.287175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=1d28b235f4434aa086cdfc0fc81c7551, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.289759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.315634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/598] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:17 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/1d28b235f4434aa086cdfc0fc81c7551/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:17.318386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.426232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938a261-91e7-43eb-bf6d-d75dc8d0fe0a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.452613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167c856-26a6-4f7c-839b-b3802934e419 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.495367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.496043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.496602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.496602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.505377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.506858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.514390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.515425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.515923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.516427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.518335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:17.518902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a3d9f2b4-9fca-49f4-9faf-3292f614ad24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:17.520298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:17.521163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:17.522050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/599] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:17.523185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139f875-678a-4556-b627-b87681f38407 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.523680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1ced-c371-43cf-970a-02777030fdf3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.528334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139f875-678a-4556-b627-b87681f38407 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.528675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139f875-678a-4556-b627-b87681f38407 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.530422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/600] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:17.531294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139f875-678a-4556-b627-b87681f38407 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.555185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/601] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:17.556660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.556959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.557245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.579501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:17.582261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:17.607685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:17.608398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:17.671692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139f875-678a-4556-b627-b87681f38407 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.672488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/602] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:17.679779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.709717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.710351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.710638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.710916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.711731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:17.712129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.713578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.726286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/603] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:17.729326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.825042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc64a3-df9c-499d-982e-74abc8ebb624 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.850756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbacf6-6b3b-46d3-9d3e-054d09367bb2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:17.857954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:17.869024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.870426 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.870946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.871470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.888382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:17.888605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:17.889184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:17.889514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:17.889799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:17.891016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:17.904851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:17.912036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:17.912995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:17.913824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:17.914452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:17.915287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:17.917088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:17.917088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:17.917088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:17.917088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:17.917088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/604] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:17.917088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.924422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/605] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:17.925771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:17.931244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:18.049919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1321b1-e1dc-4f76-bc8a-7d0f2d6f6d2f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.068873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2472d6-22b0-4064-bd88-6e50738187ac None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.088771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.089785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.090302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.090786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.091720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:18.092237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.093985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.109123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/606] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:18.110648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.113742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.114346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.114739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.115125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/0a47f6ceb2a146b092ebeb72e8679335/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.137376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=0a47f6ceb2a146b092ebeb72e8679335, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.139316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.143886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:18.159995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/607] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:17 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/0a47f6ceb2a146b092ebeb72e8679335/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:18.161057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.244310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:18.265048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af0bda-cacc-47b9-a6f9-5f7ae2fc7278 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.299703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbe43-3b82-4bf1-840b-17c642591707 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.322738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.324897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.325517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.327494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.329613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:18.332547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.335566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.350627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/608] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:18.359781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.368224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.369216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.369644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.370060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.387385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.388319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:18.414680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:18.415740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:18.416630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:18.419300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/609] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:18.424321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.495789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:18.514819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277258b-07df-411c-a15f-402a60d0dbc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.576290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3a3b7-32e5-476f-845a-51c11ee34ade None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.582218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.582934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.583268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.583579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.607845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.609335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.635309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.635926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.636258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.636546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.637318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:18.637635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:18.637974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:18.638843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:18.640188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:18.640188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:18.640188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:18.640188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:18.640409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.640489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/610] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:18.642677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:18.647898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.654175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/611] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:18.666703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.770063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271448b-85a8-4819-8a58-4920891b7498 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.814563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.815480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.815777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.816061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.817215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:18.817532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.819523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640ca20-adff-4aa3-b6b3-51fa7e813b6e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.823135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854c1e3-5597-4e21-9372-14af1bd5476d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:18.831812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:18.841240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/612] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:18.841240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918d81-68e7-4b52-a06f-0997e7f92a74 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.841240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918d81-68e7-4b52-a06f-0997e7f92a74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.841240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918d81-68e7-4b52-a06f-0997e7f92a74 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.841667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/613] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:18.848514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05918d81-68e7-4b52-a06f-0997e7f92a74 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:18.880013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ad7b8-075b-453a-a981-394e621689ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:18.881069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ad7b8-075b-453a-a981-394e621689ab None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.881452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ad7b8-075b-453a-a981-394e621689ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.882311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ad7b8-075b-453a-a981-394e621689ab None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/eac8ab3df29343c5bfa8928000f99b28/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.901954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ad7b8-075b-453a-a981-394e621689ab None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=eac8ab3df29343c5bfa8928000f99b28, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:18.903591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ad7b8-075b-453a-a981-394e621689ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:18.923785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/614] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:18 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/eac8ab3df29343c5bfa8928000f99b28/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:07:18.925511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:18.925930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:18.926288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:18.965939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:18.967339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:18.970960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05918d81-68e7-4b52-a06f-0997e7f92a74 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.015373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853422b7-b780-4734-b146-396ad25817ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.015373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853422b7-b780-4734-b146-396ad25817ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.015373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853422b7-b780-4734-b146-396ad25817ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.015373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853422b7-b780-4734-b146-396ad25817ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.022820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:19.029609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853422b7-b780-4734-b146-396ad25817ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:19.032434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853422b7-b780-4734-b146-396ad25817ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-853422b7-b780-4734-b146-396ad25817ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:19.053261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:19.054485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:19.055615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/615] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:19.066476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853422b7-b780-4734-b146-396ad25817ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.115462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/616] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:19.135560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.202645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:19.269516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426be18f-c426-47af-8996-1122b7ee3394 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.302018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853422b7-b780-4734-b146-396ad25817ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.311039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.312285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.312783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.313272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.348152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:19.353205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:19.354391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:19.355236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/617] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:19.360553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.371687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.372698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.373219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.373713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.374919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:19.375510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:19.377456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:19.400048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/618] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:19.408853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.409732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:19.579602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0461-2fa9-4df2-acbf-f694365579d4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.599766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674bdd-4727-4f03-99fa-f89bf8fb735a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.620210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:19.636079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.636732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.637017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.637296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.638028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:19.638338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:19.639785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1000a1-6f12-4693-b964-479e46b4220a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:19.640754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.641562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.641907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.642230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.653805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:19.655323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:19.658734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/619] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:19.659771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef403-000e-4f4c-b109-2c6818e38d08 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.660058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef403-000e-4f4c-b109-2c6818e38d08 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.660339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef403-000e-4f4c-b109-2c6818e38d08 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.664712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/620] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:19.667464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef403-000e-4f4c-b109-2c6818e38d08 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e109168e-5a89-4682-a5d0-e2e96a3433d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:19.672424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:19.673987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:19.674887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:19.674887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:19.674887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:19.674887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:19.674887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/621] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:19.679465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.679465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.679465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.696284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:19.697630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:19.777890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:19.781886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/622] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:19.783594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.820194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef403-000e-4f4c-b109-2c6818e38d08 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.873688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.874269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.874558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.874837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.886546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:19.888060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:19.904883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:19.905843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:19.906713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:19.906713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:19.906713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:19.906713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:19.906713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/623] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:19.907652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:19.929694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:19.935446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3f22e-e3b2-41db-a52d-cd0f24a2242a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:19.988874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:19.990471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:19.991256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:19.991797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:19.992990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:19.994435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:19.996096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.012706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/624] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:20.013572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.033785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a6cd8-5997-4640-ad5f-1f252b4013cc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.074316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:20.085576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.086534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.086974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.087607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.099945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:20.106152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:20.107274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:20.108372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:20.111272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/625] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:20.112342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.127434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56a46c-93d5-4eef-99a6-d230980598df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.176431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.177326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.177669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.177946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.179179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:20.179527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.181492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.202162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/626] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:20.203712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.231302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:20.253462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55128bd-f71b-45ef-a337-b5f799863f98 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.300890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.302547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.302992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.303848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.314393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.316382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.326905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8fa78-2ee2-4ade-8b73-93ccb82b3a95 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:20.334315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:20.335504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:20.336615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:20.339696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/627] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:20.340847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.372341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.373180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.373571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.373997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.374822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:20.375337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.377848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.391593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/628] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:20.391593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.414363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:20.475611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da049-c818-4a3d-a6f0-948201d8e9e7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.487450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:20.507602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4bb85-fdf1-4137-a996-5e5cdc99a693 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.514086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.514956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.515535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.516037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.526120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.527921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:20.541422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:20.542430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 05:07:20.543395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:20.550102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/629] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:20.550102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.552341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.553505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.554897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.555463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.556482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:20.557017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.558828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.574167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/630] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:20.575487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.698267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bb51-fc89-440d-8e06-7b204041c34f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.737772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.738410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.738708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.738986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.739849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:20.740165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.741651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.758109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:20.763036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/631] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:20.764106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.808363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8cfe-b650-4ae1-80ca-9bede40524c6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.820271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:20.850168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.851304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.851634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.851908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.863652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.865165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.877230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b33d-578f-4dc0-a4f6-29acc4de3654 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3e61b428-0f07-42f7-9775-aeb04671df38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:20.880234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:20.881301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:20.882215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:20.882215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:20.882215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:20.882215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:20.882215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/632] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:20.882623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec742f-de0b-4098-bfe1-303d20ffc9da None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.882827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec742f-de0b-4098-bfe1-303d20ffc9da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.883107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec742f-de0b-4098-bfe1-303d20ffc9da None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.888554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/633] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:20.888554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec742f-de0b-4098-bfe1-303d20ffc9da None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:20.920609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:20.921467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:20.921882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:20.922177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/1176e3df3023409b93375dc94ffb661c/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:20.938175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=1176e3df3023409b93375dc94ffb661c, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:20.939964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:20.955001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/634] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:20 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/1176e3df3023409b93375dc94ffb661c/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:20.955914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.009798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec742f-de0b-4098-bfe1-303d20ffc9da None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.044975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.060656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.061696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.062135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.062568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.114514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.131058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef2e04-0e1d-4ff4-8972-d66bc6bf0d21 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.162724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:21.164393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:21.173853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.174493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.174780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.175390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.175927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:21.176239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:21.178051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaad1a9-284d-4942-bbd3-cfc954a482fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:21.192683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/635] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:21.194072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5315c7-e5e5-4f0e-9c3a-b9a11f876200 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.194377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5315c7-e5e5-4f0e-9c3a-b9a11f876200 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.194644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5315c7-e5e5-4f0e-9c3a-b9a11f876200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.213619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a5315c7-e5e5-4f0e-9c3a-b9a11f876200 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:21.215063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a5315c7-e5e5-4f0e-9c3a-b9a11f876200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:21.291965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/636] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:21.292357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/637] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:21.293601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.293933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242e9c-5a75-4def-aceb-61f2d3693fb0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.294230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242e9c-5a75-4def-aceb-61f2d3693fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.294508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242e9c-5a75-4def-aceb-61f2d3693fb0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.311376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/638] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:21.311376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242e9c-5a75-4def-aceb-61f2d3693fb0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.348076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.369521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.411511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ee20a-f797-48b0-938e-9ee8a611ae0a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.414555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242e9c-5a75-4def-aceb-61f2d3693fb0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.447764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.448462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.448769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.449047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.450820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.451576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.452495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.452495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.457742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:21.459227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:21.464474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:21.465870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5d67b8f6-8f13-425f-8a2e-efd0841a51c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:21.472308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:21.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:21.474299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/639] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:21.475882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.475882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.475882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:21.487087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:21.488046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:21.488967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:21.488967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:21.488967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:21.488967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:21.488967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/640] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:21.493528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.495254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:21.496519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:21.572611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/641] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:21.573929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.602197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.613234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbba22f-5a88-443e-a39a-ac5bf5f531e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.645529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.667016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.667888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.668276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.668646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.681382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:21.686918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:21.688151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:21.690125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:21.690821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/642] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:21.691243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.714416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127c6bd-bd2c-4cd2-8064-a319fe783d93 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.758589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.759155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.759481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.759794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.760600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:21.760913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:21.762381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:21.781121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/643] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:21.782089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.812448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef2287-d159-4b5a-ab72-c31d8be62f4d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.850372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108989-9359-462a-a216-074794672b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.851053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108989-9359-462a-a216-074794672b90 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.851370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108989-9359-462a-a216-074794672b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.851655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108989-9359-462a-a216-074794672b90 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.852429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108989-9359-462a-a216-074794672b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:21.852740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108989-9359-462a-a216-074794672b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:21.854236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108989-9359-462a-a216-074794672b90 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:21.869949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/644] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:21.870840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108989-9359-462a-a216-074794672b90 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.886549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.895167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7f624-7934-483f-a260-f4a2dd8c3658 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:21.909198 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:21.942720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:21.943396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:21.943736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:21.944025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:21.955444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:21.961409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:21.962331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:21.963301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:21.967452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/645] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:21.968293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:21.987029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108989-9359-462a-a216-074794672b90 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.025142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.025700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.026024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.026293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.027034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:22.027381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.028790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.037846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/646] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:22.038679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.085470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ccaf1-a5c1-419f-a2a9-2429cb2c09e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.121912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.122549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.122836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.123116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.126712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.164174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660986c6-fe38-4370-b5fe-cdde9e964de2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.175747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.207979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.209461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.211729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.212389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.212671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.212957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.220345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.221910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:22.233867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:22.234767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:22.236409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/647] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:22.237449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.320289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/648] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:22.320937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.352319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe965-02bb-4a22-88d9-5e3ae4e8852c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.393716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.394353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.394492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.394669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.394919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.395677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:22.395986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.397424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.398901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.411381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/649] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:22.411381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.433625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86aa1e-b924-4ba4-b7ce-f5dcfe1c776d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.471172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.471858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.472146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.472425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.483016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.484518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:22.501812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:22.502783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:22.504105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:22.504105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:22.504105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:22.504105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:22.504105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/650] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:22.504105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.519244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f3d1d-1f49-4009-805d-ee086b93e5b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.558395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.559021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.559310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.559620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.560375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:22.560794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.562135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.575005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/651] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:22.575641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.637611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b1672-1568-4d82-ae1e-18eed8555155 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.658901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.662569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.676032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.677033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.677396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.677681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.689575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.691050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:22.706523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:22.707524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:22.708418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:22.708418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:22.708418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:22.708418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:22.708418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/652] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:22.708702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.716956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc7604-3af5-45f2-989c-1150fa31df3b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.759551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.760230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.761208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.761508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.775562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.777077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:22.794222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:22.795274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:22.796552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:22.796552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:22.796552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:22.796552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:22.796552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/653] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:22.796552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.842509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c65c3-1195-4497-8a34-fae7b7d6881a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.883029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.883708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.883984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.884261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.884961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:22.885267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.886641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.904400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/654] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:22.904400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:22.916137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bd83-16e2-4d59-97a6-4805fbc1ea14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:22.925749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.941918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:22.957187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:22.957823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:22.958108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:22.958386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:22.959186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=7c858ed2d88a43afaf34e83155629c97) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:22.959529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:22.961284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:22.968261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/655] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/role_assignments?user.id=7c858ed2d88a43afaf34e83155629c97&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:22.968993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.049892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ac9d0-4c2e-495a-a4be-b071fd259a75 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.094237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.095525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.096041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.096323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.097271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:23.097587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.099006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.103295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b63d-0d99-4ba0-84d0-ff3fced161c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.113597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/656] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:23.114472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.144376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.145025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.145495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.145638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.146292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:23.146671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.148082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06d1c0-f0e7-436a-9f05-ced72f5a1f8f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.162868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/657] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:23.164330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a383bbb-ab01-401a-960b-51c8d301c898 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.164625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a383bbb-ab01-401a-960b-51c8d301c898 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.164907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a383bbb-ab01-401a-960b-51c8d301c898 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.166322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/658] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:23.166872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a383bbb-ab01-401a-960b-51c8d301c898 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.212448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:23.242004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:23.250381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31c09-6fff-4f38-a448-2350dfdb7d69 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.287270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.287984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.288269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.288549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.297747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.298548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a383bbb-ab01-401a-960b-51c8d301c898 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.299217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:23.314849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:23.316069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/659] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:23.317916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.343293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.344079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.344489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.344812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.352797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.354264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-19939eaa-8c1e-49e3-8bbc-af399eb6c17d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:23.366428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:23.367550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:23.368459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/660] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:23.368945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.369089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.369370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.388609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:23.389822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:23.436218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a9adc-5dc5-458d-a539-ce69bb3954aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.467399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/661] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:23.467603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.482908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.483859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.484417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.484844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.494400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.496157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.497731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:23.511890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:23.518479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/662] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:23.519791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.536335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:23.588822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad5c7e-25a3-4e2f-b193-18a6c15a494f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.626343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.626912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.627201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.627516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.628320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:23.628635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.629903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e13ff-6a7f-4420-90d4-a460aab23c22 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.630108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.653375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/663] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:23.654223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.670193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.670891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.671194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.671492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.672247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:23.672565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.674784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.689333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/664] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:23.690624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.754582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:23.775019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:23.776665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b39b9-81c0-48bd-bd9c-6f9bacdbd337 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.804013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a4047-1d34-480f-8edc-819bff42fa05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.815511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.816184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.816472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.816752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.827711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:23.833849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:23.834981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:23.835896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/665] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:23.836597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.850420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.851954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.852838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.853331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:23.854575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:23.855063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:23.857566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:23.884165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/666] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:23.884165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:23.953534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32251d-30e3-49c0-a3f4-b5e69f78b986 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.988349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd892a-664e-4c75-9b1c-ad5126fa9055 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:23.994994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:23.996116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:23.996116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:23.996468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.011971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.023127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.029092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.029721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.030010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.030288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.031081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1d28b235f4434aa086cdfc0fc81c7551) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:24.031623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.032910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.039087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/667] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/role_assignments?user.id=1d28b235f4434aa086cdfc0fc81c7551&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:24.039788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.089790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.091898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.155998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f0709-add0-493e-8478-ed73bab64232 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.202714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.203461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.203757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.204453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.205229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:24.205550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.207188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.216802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/668] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:24.217693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.223733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/669] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:24.224925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.275137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.303321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.351729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e47afe-ea37-43ba-ad51-28b82c05d0dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.366760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f460e-d434-4494-a133-74ff331e2339 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.392627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.393923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.394320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.394705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.408289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.409692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.409812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.410524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.410914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.411299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:24.428512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:24.429657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:24.430771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:24.430771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:24.430771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:24.430771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:24.430771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/670] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:24.431118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.507522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.509470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.535444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.546017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa5ea-cba8-42b4-9ca7-535ee980108e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.586192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.587055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.587579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.587974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.588670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.601978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.604323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:24.622297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:24.623266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:24.624190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:24.624190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:24.624190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:24.624190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:24.626822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/671] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:24.628148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.647912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/672] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:24.647912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.764183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38332b22-6892-47a5-870a-9a1a5a939ca0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.768655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007637bc-f904-4869-940c-69adc5b63128 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.809958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.810988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.811535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.812023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.812926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:24.814161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.814425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:24.815085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:24.815405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:24.815699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:24.816409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:24.816722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:24.817127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.818106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:24.831838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/673] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:24.833456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.839405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/674] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:24.839405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:24.841649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.863785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:24.970739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9145-70f8-43e5-95d6-0362b1c30e67 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:24.975490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b293ae2-239c-4304-9d49-5d707e7e5fc6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.013891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.015088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.015633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.016203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.022994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.023711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.023984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.024264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.027141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.029119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.039899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.041412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:25.044835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:25.045764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:25.046652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:25.049553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/675] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:25.050677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:25.059549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:25.060550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:25.061466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:25.061466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:25.061466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:25.061466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:25.067032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/676] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:25.068084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.138555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.165899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.187235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e4c68-b4fc-4d0e-bc5b-f7f1381bfe03 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.225539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d9137-aaef-4515-9706-1a0fa72646d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.239786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.240461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.240744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.241017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.249419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.251113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.263005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.263663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.263949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:25.264083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:25.265033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:25.271461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:25.272185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.272185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:25.272185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.272424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/677] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:25.272424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.272572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c60668-0f3d-4fa6-8701-325f787aec4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.290085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/678] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:25.290085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6353-2dc3-4dcf-af9f-c4fce7faa254 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.290085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6353-2dc3-4dcf-af9f-c4fce7faa254 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.290085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6353-2dc3-4dcf-af9f-c4fce7faa254 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.290085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/679] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:25.290085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6353-2dc3-4dcf-af9f-c4fce7faa254 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.390551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a53c1-f435-4a32-a892-de30cca7b861 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.410588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6353-2dc3-4dcf-af9f-c4fce7faa254 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.417717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.430130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.430759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.431040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.431315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.432065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:25.432372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.433760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.442476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.449695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.450320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.450606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.450882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.451625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/680] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:25.451916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:25.452233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.452546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.453667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8a3f2-40bf-4e35-b27f-9c535d7c98e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.479407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/681] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:25.479407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.479407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.479407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.499011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:25.500244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:25.566277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09a38-3a91-455d-a653-0a57d051439e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.595373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/682] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:25.595373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.608345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.609049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.609304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.609502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.610445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=0a47f6ceb2a146b092ebeb72e8679335) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:25.611081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.612966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.619954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/683] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/role_assignments?user.id=0a47f6ceb2a146b092ebeb72e8679335&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:25.620600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.681145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.723869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7eb79-b0fb-4677-b248-b0df6c3dd821 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.748195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709db7-a155-433a-aac9-cf2143af5682 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.756816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.786714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.787722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.788162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.788687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.790167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.790938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.791325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.791758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:25.799702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.801715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:25.819480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:25.820506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:25.821450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:25.825269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/684] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:25.826050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:25.902398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:25.903908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:25.957332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bcc20-ae94-423d-a716-e8a9a52ebfdd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:25.965419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:25.997381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:25.998434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:25.998947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:25.999472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.012399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.014202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:26.030381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:26.032085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:26.033053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:26.033053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:26.033053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:26.033053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:26.033382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/685] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:26.038272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.040210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/686] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:26.041311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.087671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:26.166634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9447a-a63e-4e35-a2a3-a8da7f854449 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.202592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e0319-743e-437f-928a-fa3442b3094b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.228383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.229604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.231766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.232085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.232863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:26.233176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.236377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.250150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.251747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.252275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.252773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.255498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/687] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:26.256599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.267030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.268886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.270808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:26.287914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:26.288881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:26.289775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:26.289775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:26.289775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:26.289775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:26.296629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/688] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:26.298526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.375020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7aad8-9be9-4f93-9835-8cbb13c9eaf1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.389262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:26.409311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089e63a-750e-4b73-83cf-8a667ad5b036 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.413589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.414213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.414496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.414770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.415491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:26.415796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.417578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.434614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/689] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:26.434614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.451883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.452521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.452807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.453089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.453937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=eac8ab3df29343c5bfa8928000f99b28) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:26.454260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.455735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff35ce-38d1-4405-8102-fce0a4949c16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.462381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/690] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/role_assignments?user.id=eac8ab3df29343c5bfa8928000f99b28&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:26.463689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b7c0-a75f-451e-ae3e-37bbedf763c0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.463982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b7c0-a75f-451e-ae3e-37bbedf763c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.464263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b7c0-a75f-451e-ae3e-37bbedf763c0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.465626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/691] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:26.466172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13b7c0-a75f-451e-ae3e-37bbedf763c0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.524796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:26.580419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6077684-8732-4ff5-8bf4-c1d9206ac7cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.619136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13b7c0-a75f-451e-ae3e-37bbedf763c0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.633142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.634048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.634340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.634624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.637120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:26.637120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.638482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d195-f5b1-414c-8383-43b76893dea0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.652011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/692] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:26.656436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.656436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.656436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.669687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.670490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.670877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.671264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.671997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.674939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:26.675086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.676347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:26.704183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/693] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:07:26 2022] POST /identity/v3/users => generated 254 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:26.706115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.713750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:26.780116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/694] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:26.816647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.849217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc45e-1a60-4c86-aaf5-f3813dd896e1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:26.904407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:26.905114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:26.905401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:26.905680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:26.929437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:26.934237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:26.953135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:26.954699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:26.956303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/695] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:26.962716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:26.965119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:27.015555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b5c34-641c-4b25-88c2-3fa892260218 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:27.053470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:27.063061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:27.063976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:27.064366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:27.064746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:27.081615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:27.087082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:27.088218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:27.089234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:27.095017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/696] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:27.099291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:27.119466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056ed0c-bfae-4b27-bb40-8bf7bf078d38 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:27.192805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:27.193520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:27.194018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:27.195167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:27.195642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:27.196191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:27.197952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:27.223685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/697] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:27.229530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:27.258547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee218f-055d-4444-ba0e-82918b269b2d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:27.370938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec8a4-8d4c-42dc-a37c-96f2b58b1489 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:27.416119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:27.442695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:27.443813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:27.444331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:27.444817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:27.445841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1176e3df3023409b93375dc94ffb661c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:27.446386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:27.448135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:27.464063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/698] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:27 2022] GET /identity/v3/role_assignments?user.id=1176e3df3023409b93375dc94ffb661c&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:27.487510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:27.514800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:27.515492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:27.515772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:27.516048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:27.517448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:27.517760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:27.519506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:27.535685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/699] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:27.541380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:27.712827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8fd5-156c-4a3c-82f5-eff176288720 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:27.768204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:27.769200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:27.769653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:27.770074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:27.785108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:27.786588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eb985c3d-8547-45bf-a018-c390e3021dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:27.806075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:27.807036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:27.808010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:27.808010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:27.808010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:27.808010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:27.808010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/700] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:27.816515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:27.968049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c45d0-ea0c-4f57-b74d-7850786baa82 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:28.021229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:28.021868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.021981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cf5c7-57ce-41d3-aa01-caef5909645c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.022167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.022287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cf5c7-57ce-41d3-aa01-caef5909645c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.022443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.022595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cf5c7-57ce-41d3-aa01-caef5909645c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.023118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:28.023448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:28.024847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:28.026085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/701] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:28.031055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.031055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.031433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.044212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/702] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:28.049644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:28.084070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:28.085300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:28.108089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:28.159513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:28.221818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66dce7-63e3-4f3a-b050-dc7cebf222f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:28.244549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/703] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:07:28.271972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:28.272725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.273017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.273300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.282803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:28.284732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:28.288703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:28.305269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:28.306330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:28.307389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/704] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:07:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:28.314876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:28.412664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8b615-e632-41ff-a10b-ec12ef59caf7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:28.460432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:28.461125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.461433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.461703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.462508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:28.464180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:28.489288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:28.490304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:28.491166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/705] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Jul 29 05:07:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:28.498227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:28.544431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:28.598360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:28.606221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa8623-ff0a-4f8b-9410-ca383af45755 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:28.655723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a818ec-3812-4f8e-9d57-18c8a8776ef9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:28.701878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224361c9-8570-451d-8755-5931350fb19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:28.702532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224361c9-8570-451d-8755-5931350fb19f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.702821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224361c9-8570-451d-8755-5931350fb19f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.703102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224361c9-8570-451d-8755-5931350fb19f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.704551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224361c9-8570-451d-8755-5931350fb19f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:28.704901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224361c9-8570-451d-8755-5931350fb19f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:28.706715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224361c9-8570-451d-8755-5931350fb19f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:28.712279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/706] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 05:07:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:28.719972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224361c9-8570-451d-8755-5931350fb19f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:28.772251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:28.772815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.773106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.773389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.774210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:28.774521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:28.776113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:28.829539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/707] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:07:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:28.840503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:28.840736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:28.848276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224361c9-8570-451d-8755-5931350fb19f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:28.900563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:28.901216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.901507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.901790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.902776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:28.904027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:28.905483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f591b-b327-41f4-b296-232d301ca134 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:28.921235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/708] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 05:07:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:28.978183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb475-19f6-41c0-902d-b06c69e6ee6f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.978183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb475-19f6-41c0-902d-b06c69e6ee6f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.978183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb475-19f6-41c0-902d-b06c69e6ee6f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:28.979118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/709] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:28.986823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:28.987805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:28.989830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.019389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:29.019389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:29.073828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c04aa4-22d8-479c-b935-a28934d04f5b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:29.104149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/710] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:29.126941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:29.127816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:29.128231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:29.128613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.138028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:29.195475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:29.257587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:29.259147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:29.287725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:29.301812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f637846-eef3-4da2-bc93-85fe34ec4f54 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:29.368698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:29.369704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:29.371751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:29.372256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.373719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:29.377432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:29.402229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:29.403276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:29.404144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:29.408501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/711] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:29 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:29.414172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:29.476319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/712] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:29.479636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:29.540358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6230bd3-9103-49b3-a643-9560ff3f28fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:29.588771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:29.619005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:29.620039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:29.628665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:29.629259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.630561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:29.631111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:29.633758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:29.650810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/713] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:07:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:29.657691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:29.714738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62017e32-c163-4d64-aec6-b4beae300f0e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:29.718997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:29.828471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:29.830114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:29.830604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:29.831093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.860844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:29.862343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e1736d6-b76b-49e1-9f2c-7e253b3cf41c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:29.891464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:29.892620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:29.893504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:29.893504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:29.893504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:29.893504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:29.895902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/714] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:29.897351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865f310-643f-4b62-a980-e6cb0f84ceb7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:29.897815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865f310-643f-4b62-a980-e6cb0f84ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:29.898265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865f310-643f-4b62-a980-e6cb0f84ceb7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.903834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/715] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:29.905774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:29.906246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:29.906699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:29.926869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:29.928580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:29.940638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852f97e-be8a-457f-b83f-b48a5b951412 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:30.004064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:30.005100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.005608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.006104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.007319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:30.007893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:30.010465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:30.017016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/716] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 05:07:30.018384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.020958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/717] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:07:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:30.058337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.108449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:30.170601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf8fac-a7d8-40c1-8742-3e9b47c7a6bf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:30.221884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:30.222630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.222985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.223465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.224355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:30.224669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:30.226194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:30.242143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/718] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:30.247866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.258154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:30.283472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6049c7-aa23-4fc8-8bf3-93dbe2588e72 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:30.356982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:30.358204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.359532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.361228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.384116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:30.389939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:30.390922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:30.392760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:30.396991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/719] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:30.404199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.441180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:30.445504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcee527-75e6-4106-8628-ec31a57bf2e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:30.522675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:30.523956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.525178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.525841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.537554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:30.539725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:30.554575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:30.555576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:30.556447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:30.560308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/720] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:30.566452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.651552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce28d7-f4de-4600-8f92-baf058c544c6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:30.692437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25946-fc61-4ead-9f5b-0770ffc41b30 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:30.721632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:30.722780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.723294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.723869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.725209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:30.725760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:30.728265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144d25c-e9ca-4b86-b365-b6971cf5065d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:30.737942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:30.739999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.740512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.741377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.742367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:30.742886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:30.744695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672f9cd-42b5-41a3-aeaf-4be62b886987 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:30.745156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/721] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:30.747358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f79971-df01-4dec-9e47-5b6f0f04188c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.747834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f79971-df01-4dec-9e47-5b6f0f04188c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.748333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f79971-df01-4dec-9e47-5b6f0f04188c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.757765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/722] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:30.759367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f79971-df01-4dec-9e47-5b6f0f04188c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.761285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/723] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:30.768129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:30.768574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:30.769093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:30.772651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:30.802105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:30.804992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:30.864476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:30.923408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/724] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:30.923408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:30.954734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f79971-df01-4dec-9e47-5b6f0f04188c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.038973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f3a95-ec1d-408d-b2de-d132a1d6957e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.054561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.056405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.057333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.058091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.078746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:31.079776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.080526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.080995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.081377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.082281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1d28b235f4434aa086cdfc0fc81c7551) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:31.082699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:31.083180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:31.084616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:31.098714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/725] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/role_assignments?user.id=1d28b235f4434aa086cdfc0fc81c7551&scope.project.id=cd00b15fba514550924237d11784db71&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:31.099428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:31.117174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:31.119336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:31.121629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:31.121629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:31.121629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:31.121629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:31.125859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/726] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:31.134780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.136350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:31.287466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:31.346941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5e14d-9e0b-4e19-8f85-901aeff5179f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.358854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ee895-9ea7-4d43-8d58-9155d90ed27c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.389562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.390337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.390689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.390976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.414275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:31.425494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.426139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.426682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.426967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.427710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:31.428024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:31.428492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:31.430185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:31.432805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/727] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:31.435395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:31.445274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.458142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/728] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:31.463658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.605846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:31.633796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3184-622f-44c3-8b1b-ce7afc5a4625 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.653960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:31.654775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcce55e-25d6-4c1f-8dfc-ae3176178983 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.676144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.677079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.677366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.677623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.686497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:31.688617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:31.695594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.696239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.696524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.696801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.697537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:31.697843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:31.699253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fd12-0690-40f1-bebd-87acb4b007fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b1c526fb-496c-43f5-8279-7e321f1419a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:31.702515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:31.706467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/729] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:07:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0add-2228-4fc2-bbb1-c8ce8f232746 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.710185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0add-2228-4fc2-bbb1-c8ce8f232746 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.720104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0add-2228-4fc2-bbb1-c8ce8f232746 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.720104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/730] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:31.720104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a0add-2228-4fc2-bbb1-c8ce8f232746 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.720292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/731] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:31.720292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.720292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.720292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.738114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:31.740368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:31.827473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/732] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:31.829429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.831787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a0add-2228-4fc2-bbb1-c8ce8f232746 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:31.875971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:31.876943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:31.877070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:31.877361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:31.878187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:31.878582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:31.880189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:31.901497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/733] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 05:07:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:31.902511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:31.925008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:31.947140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:31.969221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfe9e6-828e-4617-923d-1c1ec3390161 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.010509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:32.011641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.012241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.012702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.025466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:32.026447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cd253-fb8a-4b37-95f7-44c3e45e11ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.027567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:32.044792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:32.045748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:32.046640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:32.046640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:32.046640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:32.046640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:32.049517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/734] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:32.050762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:32.067867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:32.068517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.068806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.069085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.069806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:32.072024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:32.103376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/735] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:07:31 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:32.103376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:32.282480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:32.314854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f40f9-5af9-4760-904d-81f06f9e8e9b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.317637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92663e5b-3103-4f77-9837-ef0bb432e853 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.355143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:32.380404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:32.380623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:32.381265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.381475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.381576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.381735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.381829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.381997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.382606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:32.382853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:32.384313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b0685-0687-4518-9aa6-a469d1ef9702 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:32.400716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/736] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:32.402784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca88ff-c766-4cb7-b8c4-4aeab3526d34 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.403294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca88ff-c766-4cb7-b8c4-4aeab3526d34 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.403797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca88ff-c766-4cb7-b8c4-4aeab3526d34 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.408275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/737] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:32.409650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca88ff-c766-4cb7-b8c4-4aeab3526d34 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:32.482661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:32.484564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:32.565228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca88ff-c766-4cb7-b8c4-4aeab3526d34 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.614616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:32.616026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.616542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.617021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.629000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:32.631853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6329bac9-af1b-43ef-af4d-2ca4f8cf066f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:32.657742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:32.658788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:32.659753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:32.660874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/738] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:32.662764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.663535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.663816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.676613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/739] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:32 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 05:07:32.676830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:32.691916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:32.693035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:32.696124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:32.701017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:32.775487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/740] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 05:07:32.775487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:32.825779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0df4-5128-4338-ace4-1e31454b0124 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.870739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:32.871669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:32.872095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:32.872523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:32.874076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:32.874498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:32.876674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:32.898554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/741] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 05:07:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:32.900134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:32.923580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aafb10-3c65-444f-93ea-e80ac17688aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:32.998744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:33.000299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:33.001736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:33.003606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:33.012242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:33.033074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:33.040228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:33.041475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c80b1-85ea-4934-83d2-591a58f2ac0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:33.045785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:33.074302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:33.076333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:33.079294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:33.079294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:33.079294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:33.079294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:33.083256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/742] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:33.084255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:33.116874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:33.118374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:33.120175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:33.120963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:33.123052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:33.127922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:33.167024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/743] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:07:32 2022] POST /identity/v3/services => generated 240 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:33.169458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:33.327626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472b71-df7d-41c1-90c2-502e55c9775b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:33.416564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:33.418178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:33.418581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fac17f-7972-4992-9b90-e78f0362d377 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:33.419273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:33.420316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:33.471394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:33.472825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:33.473266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:33.473694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:33.474703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:33.475165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:33.476806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:33.480161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:33.493789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/744] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:33.501670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:33.539445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:33.558284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:33.560251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:33.660155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dfe84-96ad-463c-ac4f-f1089c3c4c43 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:33.701904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:33.702855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:33.703461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:33.703951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:33.712268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:33.714251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:33.727814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:33.728782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:33.729675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:33.732414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/745] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:33.741471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:33.812376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/746] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:07:33.821174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:33.822181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:33.886700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4b108-a80b-4e41-b3c3-267054913689 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:33.934822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:33.936529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:33.937300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:33.938703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:33.940077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:33.940653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:33.942510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:33.961094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/747] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:33.973417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:34.053827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:34.090158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf980-784f-425f-869e-31b79c08d226 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:34.116280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ad29-af3d-488b-8319-a32d91a3d968 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:34.140113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:34.182189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:34.183487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.185166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.186361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.187624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=8d737070d3404633a4f6fdd25caa8964) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:34.189423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:34.193395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:34.204177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:34.207675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.209122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.210576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.213549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/748] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:33 2022] GET /identity/v3/role_assignments?user.id=8d737070d3404633a4f6fdd25caa8964&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:34.227897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:34.229789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:34.249030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:34.250088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:34.251158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:34.251158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:34.251158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:34.251158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:34.253841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/749] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:34.261668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:34.388263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb75703-e1ad-459e-b66b-708149ba0be6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:34.431564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:34.432629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.433147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.433642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.434591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:34.435142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:34.436927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:34.451666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/750] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:34.461445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:34.625343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434600d3-86b0-43f7-9762-917f37880a87 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:34.678894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:34.679995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.680510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.680996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.694175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:34.696023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:34.704895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e769b59a-542c-4e6d-aa1e-92593ec44f6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:34.715918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:34.716885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:34.717788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:34.720779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/751] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:34.726701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:34.763635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618119df-b409-4029-85e3-378a6c9325dc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.764286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618119df-b409-4029-85e3-378a6c9325dc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.764794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618119df-b409-4029-85e3-378a6c9325dc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.767375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/752] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:34.772597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d3f5c-ee50-400e-8750-0cd2ddab87a5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.773210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d3f5c-ee50-400e-8750-0cd2ddab87a5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.773713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d3f5c-ee50-400e-8750-0cd2ddab87a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.798400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:34.800567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-206d3f5c-ee50-400e-8750-0cd2ddab87a5 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:34.802427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-206d3f5c-ee50-400e-8750-0cd2ddab87a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:34.930609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917a504-8b19-4137-bd40-34b25d0ddd8e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:34.935032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/753] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:34.939044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb505d3-9b60-4c47-8665-806576de23aa None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.939396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb505d3-9b60-4c47-8665-806576de23aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.939752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb505d3-9b60-4c47-8665-806576de23aa None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.946476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/754] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:34.950312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.950648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.950873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:34.974743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:34.976008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:34.998306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:34.999342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:34.999662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:34.999994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.001816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:35.002563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:35.004141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:35.022603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/755] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:35.024618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.063236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/756] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:35.064571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.152086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:35.180709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:35.186280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c08fbb-85fe-49cd-8be6-17b33cdeae3e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:35.225442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e8694-966d-42ee-9cca-e7fbbc6ea374 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:35.244655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:35.245781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:35.246309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:35.246824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.261522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:35.270804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:35.271942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:35.272867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:35.273538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:35.273538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:35.273538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:35.273538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.274474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1176e3df3023409b93375dc94ffb661c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:35.274474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:35.276188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:35.277207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/757] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:35.278644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.290899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/758] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/role_assignments?user.id=1176e3df3023409b93375dc94ffb661c&scope.project.id=cd00b15fba514550924237d11784db71&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:35.292313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.510945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f8c95-cef6-4d50-bef4-2e4d32451159 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:35.537923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:35.551641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:35.552353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:35.552645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:35.552923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.553654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:35.553965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:35.555379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:35.559578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008077-39be-4fbc-8e4d-86d2cf842d6a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:35.566148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/759] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:35.574351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.605506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:35.606468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:35.607300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:35.607624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.608275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:35.610055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:35.623469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:35.624499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:35.625930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/760] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:35.632567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.665272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:35.800240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49961bd5-4c15-442c-8970-53c50ca8826f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:35.809376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7ac9d-f35c-461a-a48e-5f4e29870333 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:35.856939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:35.857606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:35.857891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:35.858792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.859600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:35.859922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:35.861577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:35.865074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:35.865755 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:35.866032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:35.866308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:35.879224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/761] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:35.884686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:35.886266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:35.887423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:35.922113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:35.923090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:35.924037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:35.924037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:35.924037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:35.924037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:35.924260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/762] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:35.932521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:35.995751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:36.080454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:36.100836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5739-1b08-4076-b7cc-1c22aa0d73bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:36.140223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce2a3-444e-4718-bd9e-87f48d990962 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:36.155926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:36.157152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.157438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.157725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.158427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:36.158742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:36.160208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:36.189970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/763] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:36.209513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:36.247673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:36.248784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.249082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.249348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.250060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:36.250419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:36.252196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:36.280191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/764] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:36.286056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:36.433275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74000ed5-35f2-4d7e-943a-4bf9d490a060 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:36.482180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:36.482825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.483111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.483439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.484212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] RBAC: Adding query filter params (interface=public, service_id=13bcffa37e004fec9ec96229e287217b, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:36.485055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:36.486551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62135c49-0411-4427-8577-9a27661389a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:36.506116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/765] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:07:36 2022] GET /identity/v3/endpoints?service_id=13bcffa37e004fec9ec96229e287217b&interface=public®ion_id=RegionOne => generated 194 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:36.508589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a77c5-e85f-4c9f-a417-514fa8081ac3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.508903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a77c5-e85f-4c9f-a417-514fa8081ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.509151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a77c5-e85f-4c9f-a417-514fa8081ac3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.510500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/766] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:36.514969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a77c5-e85f-4c9f-a417-514fa8081ac3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:36.542003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:36.546348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:36.552218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c458eff-c1ff-47ed-8726-8b22fb8d1c74 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:36.621925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:36.623413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.624100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.624807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.646355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:36.650704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:36.666713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a77c5-e85f-4c9f-a417-514fa8081ac3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-180ef156-9615-47a5-a7d0-82ac76a783a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:36.668883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:36.669885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:36.670849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:36.675958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/767] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:07:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:36.679920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.680995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.682441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.707589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:36.709089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:36.712447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:36.713436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.713910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.714372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.715224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:36.716845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:36.729049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/768] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:07:36 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:36.730464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:36.834827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/769] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:36.867503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:36.882242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:36.896260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b4c7e-9121-4bbf-801a-55aa1e67f687 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:36.946098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:36.947087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:36.947730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:36.948233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:36.949771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:36.950597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:36.954069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:36.980031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/770] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:07:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:36.986116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.052124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:37.072551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb0a74-294d-407b-8217-29d91245aa05 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.117222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.127629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c275291-36a1-4c75-b12b-a92f4cf4046e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.128047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.128047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.128047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.128047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.128047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-94ae89e7-930f-4f7a-9d01-b7982f431d6d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:37.131640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:37.132653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:37.133600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/771] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:36 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:37.135003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2bcd-b2ea-4281-aa33-860cba3b31c1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.135296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2bcd-b2ea-4281-aa33-860cba3b31c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.135607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2bcd-b2ea-4281-aa33-860cba3b31c1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.137502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/772] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:37.148574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.148917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.149269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.163115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.164113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.164551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.164978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.176672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:37.178342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.180130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.182129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:37.184296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:37.199740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:37.201335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:37.202870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:37.202870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:37.202870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:37.202870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:37.210689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/773] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:37.212053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.296392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/774] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:37.298582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.363606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b219aa-8eef-45cf-a5ec-13ec32b79732 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.412033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:37.414481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.415489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.415968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.416463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.417454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:37.418187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.420045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.426731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/775] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:37.428095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.437204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34d57-9faf-4d7e-b0f0-05e910940695 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.499278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.500445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.500976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.501476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.502449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:37.502979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.504669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.509783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:37.523682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/776] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:37.525012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.580421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb42cc7-e90d-4e47-a3c0-19b764635135 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.632466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.633222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.633511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.633891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.663513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fe060-7947-4ded-860a-1def2a989748 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.664245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:37.681448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:37.683381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:37.685133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:37.688829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/777] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:37.688829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.724517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.725711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.726251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.726707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.727599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:37.728098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.730926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.738075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/778] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:37.739202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.753533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:37.815232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0090-7552-477e-9eec-8ee4e860344b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.836697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:37.866787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.868185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.868741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.869302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.876928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191be2f6-546d-4a77-88b2-010f9d09edd2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:37.877109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.878473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:37.891956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:37.893015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:37.893910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/779] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:37.894399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:37.926959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:37.927636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.927910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.928206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.928906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:37.929212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:37.930604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc294d-f54c-4b1f-aa2d-2d2651e81a42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:37.939953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/780] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:37.941226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b71b1-eeec-49df-a9a4-540363565f17 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:37.941518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b71b1-eeec-49df-a9a4-540363565f17 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:37.941793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b71b1-eeec-49df-a9a4-540363565f17 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:37.946961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/781] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:37.946961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b71b1-eeec-49df-a9a4-540363565f17 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.027438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7ea94-1607-4f62-b736-5158a712b5ca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.044407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:38.069364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.070032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.070315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.070594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.071461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] RBAC: Adding query filter params (interface=public, service_id=0a94383fa7474ced8c8a6f9c70227955, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:38.071777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.073203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.079220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/782] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/endpoints?service_id=0a94383fa7474ced8c8a6f9c70227955&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:38.080142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.096775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b71b1-eeec-49df-a9a4-540363565f17 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.132006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:38.139680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.140327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.141226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.141448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.142305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:38.142618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.144131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ad37a-bdb5-46db-8f14-1eb099ccd40b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.166780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/783] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:38.166780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.166780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.166780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.190078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:38.191947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:38.229681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3321db-b65b-4738-a50e-b097430eddd6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.283429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/784] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:38.283429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.290486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.291485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.292042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.292547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.305696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.307501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:38.323126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:38.324253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:38.325270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:38.325270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:38.325270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:38.325270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:38.326802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/785] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:38.327707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.347685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:38.390076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f53259-d44b-4fad-8ad0-36f5ade38d7f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.431082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.431752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.432066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.432343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.433011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.434126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:38.434406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.441712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/786] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:38.442740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.457176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b610b-0684-461c-b2db-b3cb8ef2e318 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.501457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.502436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.503600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.504080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/1d28b235f4434aa086cdfc0fc81c7551/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.507622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=1d28b235f4434aa086cdfc0fc81c7551, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.509751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.531595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/787] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:38 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/1d28b235f4434aa086cdfc0fc81c7551/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:38.532949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.593673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45ae0f-2f7b-4778-86d8-ccbce73d57f6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.630092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:38.661670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.662475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.662763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.663046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.690727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:38.696940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:38.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:38.699434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/788] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:38.704641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.715697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01815c81-7d12-4e50-974a-81ea270ebf65 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.755870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.756949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.757451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.757983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.758905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:38.760227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.761907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.776923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/789] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:38.783175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.806390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:38.837346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8f10-d5ff-47bd-8bf7-1669c0a81845 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:38.893579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:38.894560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:38.895845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:38.896315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:38.897223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:38.898862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:38.902597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:38.921787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/790] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:38.928243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:38.959269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa576a10-59bd-4d15-b3d1-807c6d112ea1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.005456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.005993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.006270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.006550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.021785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.026506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.030842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:39.056315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:39.057351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:39.058939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/791] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:39.064417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.115995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8e07-cb72-4349-ad51-ecd4abec81a7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.155410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.157636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.158681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.159179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.159696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.174767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.177058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.194616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65aac7-da95-44d3-9472-4e87309545d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1e099f53-1ef2-43e8-8432-f523be56a314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:39.195923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:39.196813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:39.197707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:39.197707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:39.197707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:39.197707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:39.200599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/792] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:39.203042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5514e-1bdc-4860-9509-0a349c138ca7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.203042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5514e-1bdc-4860-9509-0a349c138ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.203042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5514e-1bdc-4860-9509-0a349c138ca7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.206319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/793] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:39.216544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.216544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.216544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.240227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.240483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:39.240585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.241476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.241594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:39.241679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.242723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:39.242723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.244710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.271253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/794] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:39.272443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.324852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/795] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:39.326122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.367992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.405776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4473d3c-adc6-43ec-a15e-9f5a64cab061 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.435871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.446589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.447691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.448288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.448768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.449661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:39.450167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.451142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8174e-b79b-4bf5-b34b-67b0d7642366 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.452029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.465058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/796] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:39.466283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.490801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.491535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.491827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.492110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.504511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.506037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:39.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:39.523753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:39.524634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:39.524634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:39.524634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:39.524634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:39.524634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/797] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:39.525391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.582624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e272a-eccf-47da-90dd-c533a0adf9bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.615285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.622196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de639aea-12da-470d-a613-a556acef7986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.623189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de639aea-12da-470d-a613-a556acef7986 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.623774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de639aea-12da-470d-a613-a556acef7986 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.624267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de639aea-12da-470d-a613-a556acef7986 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.625159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de639aea-12da-470d-a613-a556acef7986 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.627031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de639aea-12da-470d-a613-a556acef7986 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-de639aea-12da-470d-a613-a556acef7986 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:39.637133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:39.638074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 05:07:39.638908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:39.639732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f219-b907-4a5b-a25f-b457b0d9dd4b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.642814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/798] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:39.643872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de639aea-12da-470d-a613-a556acef7986 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.684492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.685122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.685413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.685748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.695010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.696814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.704078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:39.712430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:39.713698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:39.714788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:39.716772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/799] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:39.717779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.775617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de639aea-12da-470d-a613-a556acef7986 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.815020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.815804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.816884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.816884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.817322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:39.817771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.819323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.837871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/800] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:39.838852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.860087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572a82-9393-45c2-85be-2cae10185dc7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.911083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:39.912457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:39.913087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:39.913522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:39.914551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:39.915336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:39.917109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:39.928976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/801] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:39.930964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:39.935727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:39.965549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b55c1-b4f1-4492-b145-d50bc5a021ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:39.982388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.006756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.007919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.007919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.008348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.009306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:40.009769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.011264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.030922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/802] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:40.037062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.046412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea76d-7757-475b-8c94-e71abdf47c61 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.083530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.084441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.084963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.085406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.092900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.095516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:40.108415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:40.109825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:40.110967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:40.113118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/803] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:40.114101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.152892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb5f6-5089-405e-8444-45c396e619cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.188759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.191423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.192081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.192370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.192647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.193396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:40.193708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.195154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.206072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/804] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:40.207049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.240945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ebe50-3b1e-490b-acfc-e091285617a7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.244378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.303134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.304235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.304721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.305185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.317634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.319473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.330849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530188-6290-438f-a422-667d15aad0d4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:40.336780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:40.337731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:40.338787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:40.338787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:40.338787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:40.338787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:40.341332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/805] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:40.342530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.371551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.372195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.372758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.372758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.373504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:40.373817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.375990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.389032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/806] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:40.390121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.459984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64646b46-c5dd-4f2c-a5ec-ef40051d4faf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.477259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.502731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.503948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.504454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.504941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.505909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.507580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.534966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/807] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:07:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:40.536235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.564854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.580193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120a3e0-d094-4eaf-8007-f015287aa2d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.634504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.635141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.635454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.635738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.636748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:40.636748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.638175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.652664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/808] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:40.661678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983bbb-ab0b-4a0a-b060-c1c38a134fe9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.661908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.703828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b7357-b417-464e-b13f-cac0a509a302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.705279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b7357-b417-464e-b13f-cac0a509a302 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.705777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b7357-b417-464e-b13f-cac0a509a302 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.706268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b7357-b417-464e-b13f-cac0a509a302 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/8d737070d3404633a4f6fdd25caa8964/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.725768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b7357-b417-464e-b13f-cac0a509a302 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=8d737070d3404633a4f6fdd25caa8964, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.727706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b7357-b417-464e-b13f-cac0a509a302 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.745589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/809] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:40 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/8d737070d3404633a4f6fdd25caa8964/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:40.790430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654315-ee3a-485f-82f0-276eaf64283f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.790896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654315-ee3a-485f-82f0-276eaf64283f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.791334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654315-ee3a-485f-82f0-276eaf64283f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.794253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/810] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:07:40.804309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.804763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.805202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.844736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:40.846201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:40.849781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbb99f-b80c-4422-8e89-b7d2719c1359 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:40.864367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.906455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:40.908277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:40.908726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:40.909158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:40.923551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:40.925362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:40.944805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:40.945750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:40.946638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:40.949094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/811] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:40.952006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/812] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:40.955991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:40.973186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:40.978721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.096640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598810d-e86c-449d-b55a-de9a4bbd916e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.109327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15a05a-578e-4b89-981b-9772b0b5b583 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.150928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.151721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.152009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.152295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.153270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.154751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:41.156104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.158460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.159016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.159895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.161434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:41.161950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.163696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-80789a04-a1b2-40ac-83d8-ffd71640006b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:41.171126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:41.172625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:41.173467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/813] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:41.174794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ad21-d80d-4581-abd8-b8e7a6d7ac20 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.174794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ad21-d80d-4581-abd8-b8e7a6d7ac20 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.175033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ad21-d80d-4581-abd8-b8e7a6d7ac20 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.177016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/814] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:41.182238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.182796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.183258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.192313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/815] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:41.193821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.233207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:41.236612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:41.258123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:41.294476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:41.336207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb9b70-34e4-4780-869b-f514c6bfea0d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.343414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/816] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:07:41.343414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.403444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.404401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.404872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.405331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.406976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:41.407665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.409259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:41.415651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/817] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:07:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:41.416510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.490559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da5249-df0e-403d-8bd6-bdb8887c3156 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.530046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.530743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.531031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.531585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/1176e3df3023409b93375dc94ffb661c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.533510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=1176e3df3023409b93375dc94ffb661c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.534969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:41.554701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610153a-7020-4a21-889d-04ae21227d08 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.559506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/818] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:07:41 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/1176e3df3023409b93375dc94ffb661c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:07:41.560561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.565540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:41.603405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.603405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.603405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.603405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.603888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:41.615093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:41.620289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:41.621443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:41.622295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/819] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:41.632444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.713645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b953-f981-4dc0-a028-ede60718c823 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.756872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.757798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.758307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.758793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.759759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:41.760290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.761966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:41.769422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/820] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:07:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:41.775482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.852411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7fe2f-be0f-4c44-a31b-f348d0f0bfca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.859953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:41.900186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.900832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.901120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.901395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.902117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:41.902426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.903069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aac3a4-fa48-4a42-8838-814c88e4d6bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:41.903866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:41.915666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/821] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:41.921119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:41.966358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:41.967621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:41.968066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:41.968961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:41.970177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:41.974127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:42.009444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/822] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:07:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:07:42.010718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:42.111389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56599c76-efca-4a3b-be9c-cfd63059cca9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:42.194803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:42.196021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:42.196450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:42.196840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:42.221553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:42.224812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:42.259140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:42.260215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:42.261119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:42.261119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:42.261119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:42.261119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:42.261802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/823] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:42.270514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:42.497617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049229-f353-41d8-9fa3-94cdb1642181 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:42.543426 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:42.568780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:42.569867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:42.570365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:42.570861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:42.572296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:42.572820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:42.575111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:42.597517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/824] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:42.606701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:42.818822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c32b5-c244-4a60-ba70-1cf39915cc23 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:42.823411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:42.885721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:42.886797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:42.887288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:42.887806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:42.902728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:42.905393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:42.928100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:42.929114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:07:42.930071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:42.930606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/825] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:07:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:42.940451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:43.133791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2ac42-f16c-47f2-a0d5-daeb0f89450b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:43.200509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:43.201429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.201900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.202379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.204594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:43.205158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:43.207738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de05748-0a35-49d0-ae6e-2824611c8eac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:43.243812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/826] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:07:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:43.252455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:43.334615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbcbe9-2f8b-409a-bd1f-3a59dc57fe5b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.335115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbcbe9-2f8b-409a-bd1f-3a59dc57fe5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.336254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbcbe9-2f8b-409a-bd1f-3a59dc57fe5b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.339766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/827] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:43.346344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.346631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.346904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.369377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:43.370931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:43.423415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:43.451478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb06b4-9806-4908-9d2b-c5eb62a75a56 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:43.487373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/828] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:07:43.498464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:43.499645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.500672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.501171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.511677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:43.550847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:43.600141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:43.601647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:43.674487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214e79b-e83f-4796-bc13-000a44c02e09 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:43.725490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/829] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:07:43.726322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:43.727243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.727752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.728184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.734567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:43.742348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:43.747687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:43.748882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:07:43.750676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:43.753260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/830] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:43.763582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:43.830776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:43.833908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:43.886817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532b6a2-40dd-46f4-b9c8-615f0a2157eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:43.933382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:43.935219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.935951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.936437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.954950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21615939-4c73-4efa-b21b-fa886d1a4686 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:43.962321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:43.965591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:43.995866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:43.996496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:43.996777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:43.997064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:43.997794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:43.998161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:43.999546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d784b4-fb1f-4987-b06c-5ba8edc805e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fa6844a0-e85b-4946-87f9-a5678e638c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:44.000179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:44.001974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:44.003712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:44.003712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:44.003712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:44.003712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:44.007518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/831] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:44.008918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357a099-158a-4203-9987-150f3f0d2420 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.009210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357a099-158a-4203-9987-150f3f0d2420 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.009599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357a099-158a-4203-9987-150f3f0d2420 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.012074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/832] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:44.013088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357a099-158a-4203-9987-150f3f0d2420 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.018929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/833] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:07:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:44.021377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.021682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.021950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.045903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:44.048222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:44.131735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/834] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:07:44.132634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.177337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:44.255038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357a099-158a-4203-9987-150f3f0d2420 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.283600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:44.321453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734825-f358-4529-9ff2-c44943401791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.322459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734825-f358-4529-9ff2-c44943401791 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.322936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734825-f358-4529-9ff2-c44943401791 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.323506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734825-f358-4529-9ff2-c44943401791 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.324330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734825-f358-4529-9ff2-c44943401791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:44.324750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734825-f358-4529-9ff2-c44943401791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.326253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734825-f358-4529-9ff2-c44943401791 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.343303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/835] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:44.344926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734825-f358-4529-9ff2-c44943401791 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.349577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c05d1-7626-4088-be2d-7ed2793903e0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.413882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.414981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.415485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.415984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.434689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.436474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:44.452656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:44.453671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:44.454564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:44.454564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:44.454564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:44.454564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:44.454800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734825-f358-4529-9ff2-c44943401791 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.454875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/836] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:44.455375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.496486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.497147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.497494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.497772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.498431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.500388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5509b8d1-b146-401c-9017-558d41684994 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:44.510379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:44.512310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:44.513945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/837] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:44.515606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8940a-af92-472a-9c72-0f99765a7d23 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.515606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8940a-af92-472a-9c72-0f99765a7d23 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.515606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8940a-af92-472a-9c72-0f99765a7d23 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.515606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/838] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:44.515606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8940a-af92-472a-9c72-0f99765a7d23 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.553183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:44.566919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:44.582554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe61fe3-2d72-4f82-8d92-6afbcc53d0c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.625614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.626648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.627081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.627579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.639063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.642208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.643478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8940a-af92-472a-9c72-0f99765a7d23 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-88215c5b-7e92-47a6-be2c-cdbe8d7eaa0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:44.655790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:44.658905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/839] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.660682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.661849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.682360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:44.683623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:44.690605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.691314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.691696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.692069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.692929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:44.693267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.694796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.713256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/840] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:44.714413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.778726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/841] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:44.779638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.869365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:44.874344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93470166-9899-4573-bdac-f3ac28e64408 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.884032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:44.921186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.922617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.923151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.923505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.924214 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:44.924525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.925947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.927080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13080-fb85-4652-9fa7-521260b7b478 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:44.931412 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/842] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:44.932380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:44.968256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:44.969686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:44.969966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:44.970341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:44.971265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:44.971847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:44.973534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:44.987638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/843] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:44.988975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.066607 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521ec28-e95e-4f6a-8164-6c7761802d86 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.110613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988643af-de0c-4eb2-a84c-d20406ba1e7b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.115162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.115879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.116170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.116455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.125560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.127076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:45.142120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:45.143274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/844] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:45.145137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.161762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:45.174793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:45.178764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.179834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.180764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.180764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.199372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:45.209444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:45.211471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/845] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:45.213209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.294788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ede2a-899e-48bb-baba-b4a51f40311c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.347275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.348056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.348414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.348846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.349675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:45.350050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.352030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.359828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/846] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:45.361266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.365090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4887b-44cf-495d-a575-6f0a01aa2b1d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.427696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.428685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.429137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.429584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.431020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1d28b235f4434aa086cdfc0fc81c7551) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:45.431582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.434039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.455390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/847] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/role_assignments?user.id=1d28b235f4434aa086cdfc0fc81c7551&scope.project.id=cd00b15fba514550924237d11784db71&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:45.455390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.518014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:45.543732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb91501-866a-4079-b5dd-0207a2f812f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.546716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:45.597972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.599013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.599552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.600063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.601232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:45.601811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.603578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.632493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/848] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:45.638033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.643886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb85902-7656-4f46-b5bb-64a42e4d7650 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.686672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.687899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.688410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.688910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.689645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:45.689955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.692307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.704535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/849] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:45.712598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:45.787085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da5567-2acd-4f56-91c0-c2235201aa61 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.828722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.829621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.830073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.830524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.838776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd536d-5e93-4813-a632-a487d89baee8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:45.849294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:45.857856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:45.888084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:45.889303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:45.889942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:45.890305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:45.906802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.908852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:45.926659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:45.927737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:45.928662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:45.928662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:45.928662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:45.928662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:45.931111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/850] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:45.939411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:45.939411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:45.941031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:46.127578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/851] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:07:46.132978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:46.136711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553ffc2-7d3e-4813-828c-18ea75e65143 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:46.194818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:46.195840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:46.199693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:46.199693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:46.199693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:46.199693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:46.201065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:46.230319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/852] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:46.237484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:46.256481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:46.298415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:46.381849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea6471-91fb-4b72-860b-d195c8137c94 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:46.449077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:46.450814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:46.450814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:46.450814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:46.475598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e07084-b8ab-46ec-95d1-f51d64cf2ee2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:46.478670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:46.483218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:46.519720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:46.521425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:46.523125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:46.523125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:46.523125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:46.523125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:46.526112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/853] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:46.532399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:46.555312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:46.557807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:46.558585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:46.559766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:46.581244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:46.584099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:46.608476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:46.610874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 05:07:46.612695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:46.618118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/854] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:07:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:46.620800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:46.741824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:46.751308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa9cc7-6b14-48a7-97e1-1596fe01a2a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:46.791824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:46.825251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde1806-24b6-4fc4-bfe7-afeec16aa771 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:46.835400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:46.836591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:46.836970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:46.837413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:46.838702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:46.839269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:46.842054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:46.865924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/855] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:46.872071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:46.904398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:46.906181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:46.906937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:46.907635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:46.909620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:46.910911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:46.913938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:46.956466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/856] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 05:07:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:46.965149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:47.088798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7027b10-bdd7-4bdf-aea2-dbcad9924d1e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:47.162318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:47.163306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.163837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.164320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.178475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:47.180078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564a6b7-559a-4985-821a-03a0e1e13ff3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:47.181223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:47.205978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:47.207774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:47.209973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/857] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:47.216589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:47.257599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:47.258554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.258947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.259328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.261744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:47.278540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:47.354791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:47.356355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:47.440123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e51c3-77b8-423e-b953-a373c364a503 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:47.483026 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:47.483966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.484389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.485038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.485980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:47.486787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:47.488025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:47.504340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/858] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:47.512954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:47.550049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/859] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 05:07:47.552281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:47.601489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:47.631439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:47.661611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9704b-87cf-4cbc-86fa-884846a6d208 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:47.682941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62899994-97c9-4532-aaee-516e87df1373 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:47.707076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:47.707894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.708211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.708561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.709472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=8d737070d3404633a4f6fdd25caa8964) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:47.709862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:47.711795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:47.720819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/860] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:47 2022] GET /identity/v3/role_assignments?user.id=8d737070d3404633a4f6fdd25caa8964&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:47.732792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:47.734520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.735035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.735645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.749348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:47.752248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1f9b07d6-fd50-472b-9269-d05502f145d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:47.770560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:47.771635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:47.772571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:47.772571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:47.772571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:47.772571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:47.779464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/861] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:47.781355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:47.894722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e192b2-6296-45ef-aa0d-e26232d7a9d5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.895193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e192b2-6296-45ef-aa0d-e26232d7a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.895696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e192b2-6296-45ef-aa0d-e26232d7a9d5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.905291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/862] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:07:47.908349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc186-e0be-4a71-8ece-4fd12a0def7d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.908857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc186-e0be-4a71-8ece-4fd12a0def7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.909355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc186-e0be-4a71-8ece-4fd12a0def7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.937784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3dbfc186-e0be-4a71-8ece-4fd12a0def7d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:47.940040 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3dbfc186-e0be-4a71-8ece-4fd12a0def7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:47.941508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ae516-b1bb-4ec4-8a15-0f37bbf6e590 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:47.993647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:47.994435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:47.994598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:47.994891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:47.995619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:47.997291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:47.999054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54b4b7-dd11-4c4f-9882-ac445d70a3b9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:48.017217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/863] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:48.018493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26803fd-3557-473f-ad68-9d967b975c30 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.018795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26803fd-3557-473f-ad68-9d967b975c30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.019069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26803fd-3557-473f-ad68-9d967b975c30 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.020769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/864] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:48.025935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26803fd-3557-473f-ad68-9d967b975c30 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:48.029292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:48.068678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/865] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:48.070959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.075576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.076137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.104113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:48.106946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:48.156850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:48.180378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26803fd-3557-473f-ad68-9d967b975c30 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:48.241857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:48.242526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.242822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.243098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.251812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:48.253377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:48.254124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/866] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:48.256347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:48.279721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:48.282040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:48.283878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:48.294497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/867] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:48.297383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:48.408822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:48.448797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c424ae-5728-4997-99ef-c03b0e73241f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:48.473805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b7ae2-ff1c-4915-bf28-19ca19642332 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:48.511189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:48.512286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.512794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.513302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.514634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:48.517910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:48.535988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:48.536631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.536926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.537204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.537974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:48.538333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:48.539894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:48.543325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/868] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:07:48 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:48.544724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:48.578744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/869] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:48.580726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:48.582478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:48.736852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181f1f-c135-4e9e-9fbd-90434adf5458 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:48.773364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec66c73-2f83-4dfe-87fa-824b613b7ac8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:48.781181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:48.813100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:48.814396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.814905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.815989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.817059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1176e3df3023409b93375dc94ffb661c) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:48.817614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:48.820626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:48.834230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/870] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:48 2022] GET /identity/v3/role_assignments?user.id=1176e3df3023409b93375dc94ffb661c&scope.project.id=cd00b15fba514550924237d11784db71&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:48.844138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:48.845547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:48.846185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:48.846810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:48.848411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:48.851114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:48.870058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:48.871310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:48.872412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:48.875979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/871] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Jul 29 05:07:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:48.882271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:49.015762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dbed0-9406-49c6-985b-3bbafdef4446 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:49.059038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:49.059761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:49.060009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:49.060288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:49.060984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:49.061291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:49.063738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230a17-2251-48d3-afd4-b44371d4d021 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:49.069665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/872] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:49.075415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:49.163365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:49.197368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c44e4-0237-4afc-871d-1047665a913c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:49.247376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:49.247376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:49.247376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:49.247376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:49.247822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:49.247822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:49.250307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf5fec-ba35-4f39-b87f-d3fde719dc39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:49.263792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/873] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:50.141781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97167a09-4ff7-489d-8ce9-75c24994add2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:50.142614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97167a09-4ff7-489d-8ce9-75c24994add2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:50.142614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97167a09-4ff7-489d-8ce9-75c24994add2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:50.144050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/874] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:50.149940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:50.150319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:50.150565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:50.178745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:50.181299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:50.308387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/875] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:07:50.333359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97167a09-4ff7-489d-8ce9-75c24994add2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:50.405075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:50.506187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97167a09-4ff7-489d-8ce9-75c24994add2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:50.558763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:50.559521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:50.559841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:50.560121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:50.574267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:50.580529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:50.581509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:07:50.582369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:50.586989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/876] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 05:07:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:50.588584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:50.698187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:50.770076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a06ac-ee9a-4a01-9ab5-7938ed73572f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:50.845790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:50.846915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:50.847620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:50.848381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:50.849818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:50.850346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:50.853018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d487b-b609-4fd6-a164-bf2bd7eb7722 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:50.870585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/877] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:07:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:50.877145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.083128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a1533-74e8-4190-8465-a2d415dd2926 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:51.091771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480eb8-ccfa-497b-a02c-68a27d3452ac None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.091771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480eb8-ccfa-497b-a02c-68a27d3452ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.091771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480eb8-ccfa-497b-a02c-68a27d3452ac None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.091771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/878] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:51.100746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:51.101628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.101628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.105922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.132952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:51.134644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:51.151908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:51.153013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.153458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.153924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.186872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:51.189602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:51.218974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:51.220010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:51.220903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:51.220903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:51.220903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:51.220903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:51.223413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/879] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:51.229012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.267287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/880] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:51.296095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.398131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77637ecf-a25c-4d27-95f3-3e26715730a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:51.412180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:51.465615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9195c0-f7fe-40be-b1d7-0e3abae2518d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:51.474689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:51.476441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.477244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.478171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.479743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:51.480284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:51.482900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700bf88-469f-4d79-95db-58fe0c224b30 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:51.505759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/881] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:51.510452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:51.510452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.510452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.510829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec3e1-b98e-44b9-ac77-3a727a718515 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.510905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.510983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec3e1-b98e-44b9-ac77-3a727a718515 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.511067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:51.511144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec3e1-b98e-44b9-ac77-3a727a718515 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.511238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:51.511668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/882] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:51.513069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ec3e1-b98e-44b9-ac77-3a727a718515 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cebaecbb-b600-42fb-9140-f1c1f852ed46 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:51.526930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:51.528218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:51.529088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/883] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:51.529881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.530465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.530718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.552665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:51.553999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:51.563917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:51.651952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/884] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:51.652762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.701042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ec3e1-b98e-44b9-ac77-3a727a718515 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:51.731176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:51.744332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:51.744996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.745282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.745563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.754910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:51.756430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3c7bb7fd-7a23-47c6-967b-73a850bc5464 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:51.771148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:51.773373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/885] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:07:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68170d6-c772-4b64-a78b-29b8aeae71ee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.775577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68170d6-c772-4b64-a78b-29b8aeae71ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.777433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68170d6-c772-4b64-a78b-29b8aeae71ee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.777433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/886] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:51.777433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68170d6-c772-4b64-a78b-29b8aeae71ee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.797216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525070e-d24a-4165-b4ba-c592966ef6c7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:51.848533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:51.849883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.850444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.851068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.852611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:51.853187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:51.855868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:51.871006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/887] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:07:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:51.871006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:51.874091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:51.901073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68170d6-c772-4b64-a78b-29b8aeae71ee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:51.944930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5837f6-df61-4e03-bc4a-093511944e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:51.945612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5837f6-df61-4e03-bc4a-093511944e90 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.945942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5837f6-df61-4e03-bc4a-093511944e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.946271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5837f6-df61-4e03-bc4a-093511944e90 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.947054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5837f6-df61-4e03-bc4a-093511944e90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:51.948765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5837f6-df61-4e03-bc4a-093511944e90 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:51.966033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/888] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:07:51 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:07:51.967319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:51.967635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:51.967921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:51.999778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:52.001329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:52.024908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b93d57-2fed-4bc8-8c2f-6e20f4126499 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.062766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:52.115394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/889] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:52.115394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:52.119913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:52.121120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:52.121636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:52.122140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:52.123885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:52.126098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:52.128037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:52.169398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/890] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 05:07:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:52.171636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:52.238288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:52.288677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0c97-4350-4ba8-80bf-7a7ea07a1083 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.314884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45996a06-7e05-4b30-a7e4-9702b5afc328 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.334418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:52.335241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:52.335635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:52.335987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:52.336915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:52.337305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:52.339023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:52.345912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/891] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:07:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:52.347402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:52.369388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:52.370380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:52.370814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:52.371239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:52.372258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:52.373971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:52.384751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:52.386932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:52.388878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/892] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:52.397744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:52.473151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:52.566077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f57070-b106-4f18-bfbe-430f7c05fa2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.604342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987cc02-831c-4339-a576-6a9511670412 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.607837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:52.614584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:52.615430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:52.615826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:52.616210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:52.682619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:52.685595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:52.686355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:52.687245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:52.689089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:52.691307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:52.694339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:52.707417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:52.707417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:52.711626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/893] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:52.713564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:52.829152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/894] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:07:52 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:07:52.835124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:52.870668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:52.907210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:52.912697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f2560-8e28-432d-8b8c-246d08be2290 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.958698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02490f-c14f-4bf7-820a-498da6e528cd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:52.981498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:52.982708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:52.983329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:52.984557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:52.985960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:52.986598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:52.989957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.007839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/895] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:53.009989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:53.013407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:53.014101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:53.014381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:53.014660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:53.028560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:53.030522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:53.053538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:53.055279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:53.057036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:53.057036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:53.057036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:53.057036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:53.059876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/896] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:53.061343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:53.175004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:53.217337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0756-3f11-4159-ae79-cf444f17c22f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:53.222359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb42487-d2cf-4562-ac32-35d3f86d4b6e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:53.282122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:53.283137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:53.283137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:53.283439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:53.284099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:53.284408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:53.285494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:53.285829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.287148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:53.288506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:53.290386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:53.292291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] RBAC: Adding query filter params (interface=public, service_id=4812e7c6f82f42dbb98f4255d0e44730, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:53.293158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:53.298957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.302538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/897] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:53.308037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:53.318455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/898] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:07:53 2022] GET /identity/v3/endpoints?service_id=4812e7c6f82f42dbb98f4255d0e44730&interface=public®ion_id=RegionOne => generated 194 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:53.323787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:53.371022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:53.463699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28474624-9621-4fa6-bd00-6bd790af0bb6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:53.521956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a88e-7ee8-4f42-a4d5-98973bdb3a47 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:53.549851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:53.551180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:53.551708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:53.552190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:53.570780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:53.571983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:53.572168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:53.572676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:53.573154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:53.574040 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:53.574952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.576265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.599809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/899] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:07:53 2022] GET /identity/v3/services => generated 1268 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:53.602375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:53.603323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:53.604238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:53.605963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/900] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:53.611877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:53.826811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:53.845672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8418e04-16fd-40c4-93d4-b6e7261c62ed None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:53.914702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:53.915893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:53.916393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:53.916884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:53.918301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:53.918861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:53.922288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605052cc-baa9-499c-9e13-21e832effd9a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:53.948798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/901] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:53.955376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:53.961233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:54.177255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64d8d-2edc-43d4-a03a-999e393035f5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:54.224933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000ac7-46fb-4adc-bdba-195e87684531 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:54.225695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000ac7-46fb-4adc-bdba-195e87684531 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:54.226208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000ac7-46fb-4adc-bdba-195e87684531 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:54.231215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/902] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:54.240961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:54.242017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:54.242606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:54.245605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:54.246269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:54.246559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:54.246840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:54.247722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] RBAC: Adding query filter params (role.id=644cadd7a0a0442493993029a8b2a2b6, scope.project.id=cd00b15fba514550924237d11784db71, user.id=8d737070d3404633a4f6fdd25caa8964) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:54.248039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:54.250979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:54.259509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/903] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:07:53 2022] GET /identity/v3/role_assignments?user.id=8d737070d3404633a4f6fdd25caa8964&scope.project.id=cd00b15fba514550924237d11784db71&role.id=644cadd7a0a0442493993029a8b2a2b6 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:54.278899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:54.281357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:54.427039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/904] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:54.466583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:54.601285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:54.673386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d645a-da51-4762-8f96-abd22f6cfe2d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:54.741809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:54.742869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:54.743413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:54.743932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:54.745214 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:54.748103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:54.767473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:54.768510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:54.769354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:54.770145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/905] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Jul 29 05:07:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:54.779485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:54.947870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:54.981724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80fa3e-f2c8-4d90-a945-d566636e36a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:55.054878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:55.055951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:55.056463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:55.056963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:55.058242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:55.058801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:55.061650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:55.075515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/906] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:55.080888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:55.292146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf024e-e04e-4eb8-a2f5-b57bf654af1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:55.355679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:55.365923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:55.367010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:55.367552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:55.368069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:55.369364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:55.369929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:55.372779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:55.383244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/907] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:55.393329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:55.596872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588d291-f240-4a2b-9b94-16e6ef99283d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:55.666123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:55.667206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:55.667748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:55.668250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:55.669730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] RBAC: Adding query filter params (interface=public, service_id=9f1f358f950c4d79ab4389921d9a7579, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:55.670304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:55.673369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc5c51-9115-42ae-88c9-f59c58beb384 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:55.684477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/908] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:07:55 2022] GET /identity/v3/endpoints?service_id=9f1f358f950c4d79ab4389921d9a7579&interface=public®ion_id=RegionOne => generated 194 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:55.693980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:55.828558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a82b4-396b-4422-8aa5-4aa9f451a517 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:55.830110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a82b4-396b-4422-8aa5-4aa9f451a517 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:55.831185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a82b4-396b-4422-8aa5-4aa9f451a517 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:55.835399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/909] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:55.843651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf313db-015f-42cf-a026-92645a06961a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:55.844652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf313db-015f-42cf-a026-92645a06961a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:55.845440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf313db-015f-42cf-a026-92645a06961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:55.879694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aaf313db-015f-42cf-a026-92645a06961a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:55.881229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728dc91-4733-4205-b70e-ab4282019045 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:55.882596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aaf313db-015f-42cf-a026-92645a06961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:55.903287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:55.922835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de6c2d-55dc-4468-9617-d2ddcda71ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:55.923858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6c2d-55dc-4468-9617-d2ddcda71ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:55.924355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6c2d-55dc-4468-9617-d2ddcda71ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:55.924875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6c2d-55dc-4468-9617-d2ddcda71ae2 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:55.926017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de6c2d-55dc-4468-9617-d2ddcda71ae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:55.927689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de6c2d-55dc-4468-9617-d2ddcda71ae2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:55.942156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/910] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:07:55 2022] GET /identity/v3/services => generated 1268 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:07:56.004799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feb8aa-f809-4004-ba1f-0ffc27ef414f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:56.005432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feb8aa-f809-4004-ba1f-0ffc27ef414f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:56.005869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feb8aa-f809-4004-ba1f-0ffc27ef414f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:56.007560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/911] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:56.016547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:56.016825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:56.017111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:56.038239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:56.039950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:56.043288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/912] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:07:56.087862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf313db-015f-42cf-a026-92645a06961a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:56.121719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:56.137491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/913] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:56.177355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:56.275828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf313db-015f-42cf-a026-92645a06961a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:56.334589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82808c27-4832-4b3e-a34d-3cf2a8490786 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:56.352284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:56.353435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:56.353859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:56.354344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:56.355530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:56.358461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:56.377517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:56.378552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:56.378894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:56.379892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 05:07:56.380750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:56.382237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:56.382754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:56.383758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:56.385504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:56.387282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/914] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:56.393272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:56.400110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:56.410801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/915] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:07:56 2022] POST /identity/v3/services => generated 241 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:07:56.597665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c31895-2ab8-478a-9593-cac9532ab296 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:56.678333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:56.679069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:56.679474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:56.679948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:56.681043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:56.681423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:56.682928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:56.691203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/916] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:56.696986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:56.757094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:56.818542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee03f8f-5d65-4ced-a303-9c00c423f712 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:56.859036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:56.859706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:56.860051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:56.860340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:56.861049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:56.861365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:56.862807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4401-34e3-47c5-937a-715c4355eb76 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:56.869039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/917] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 05:07:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:56.875578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:56.957299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:57.007554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1812be-93d5-403f-a3dc-bc314077f8a6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:57.052384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7636f08-95b8-43e0-9d6c-6b3eb01c5908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:57.053037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7636f08-95b8-43e0-9d6c-6b3eb01c5908 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:57.053327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7636f08-95b8-43e0-9d6c-6b3eb01c5908 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:57.053611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7636f08-95b8-43e0-9d6c-6b3eb01c5908 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:57.054363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7636f08-95b8-43e0-9d6c-6b3eb01c5908 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:57.055827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7636f08-95b8-43e0-9d6c-6b3eb01c5908 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:57.080814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/918] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:07:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:07:57.151771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ccc73-b634-4899-831c-98cdc52fc734 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:57.152458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ccc73-b634-4899-831c-98cdc52fc734 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:57.153002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ccc73-b634-4899-831c-98cdc52fc734 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:57.155227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/919] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:07:57.164524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:57.165012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:57.165506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:57.201599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:57.203988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:57.297863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/920] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:57.320498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ccc73-b634-4899-831c-98cdc52fc734 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:57.445423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ccc73-b634-4899-831c-98cdc52fc734 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:57.462845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:57.487406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:57.488474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:57.488968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:57.489450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:57.503144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:07:57.509000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:57.509997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:07:57.510836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:57.513877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/921] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 05:07:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:57.523665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:57.576547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:57.641796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4af9df-78ca-4bdd-aab8-ed2083ef545d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:57.681149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:57.682170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:57.682662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:57.683171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:57.684148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:57.684677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:57.686347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:57.696758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/922] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:07:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:57.703144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:57.838917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251874f9-fd8e-4967-b25d-45e16a088f79 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:57.902822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:57.903783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:57.904210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:57.904639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:57.925487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:57.928459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:57.957393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:57.958440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:57.959326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:57.959326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:57.959326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:57.959326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:57.960025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/923] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:57.966171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:58.132727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189bf99-cd1e-486c-8ef5-972c70cfaf97 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:58.157628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:58.203083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:58.203764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:58.204051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:58.204333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:58.205052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:58.205364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:58.206843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:58.222264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/924] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:58.228104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:58.263085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:58.409471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1678a28-dd6c-4f6f-b561-9e48442bb2a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:58.479264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:58.480334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:58.480842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:58.481335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:58.497497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:58.500918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:07:58.524935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:58.525965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:07:58.526919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:58.527998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/925] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:07:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:07:58.534094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:58.736672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd1f35-eea1-4057-9f7f-b47111c8eabe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:58.804716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:58.805717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:58.806223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:58.806707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:58.808177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:58.808717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:58.812022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ff4b-715a-441e-a0c5-c749985f8cef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:58.849453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/926] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 05:07:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:58.856478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:58.937736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839900db-856f-402d-ac30-77dab1710b51 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:58.938897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839900db-856f-402d-ac30-77dab1710b51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:58.940553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839900db-856f-402d-ac30-77dab1710b51 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:58.950915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/927] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:07:58.954916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:58.955921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:58.956676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:58.990002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:58.993453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:59.047982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:59.072362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:59.076658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aa1cc-d3c5-48c8-87db-8cc569ac39ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:59.156982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/928] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:07:59.179171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:59.180798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.181724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.182491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.196174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:59.211865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:59.217189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:07:59.263936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:59.266226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:59.268029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:07:59.268029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:07:59.268029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:07:59.268029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:59.270242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/929] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:59.280868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:59.413512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6f767-d08c-4b41-b9e6-19b2f29eaf45 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:59.459408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc6306-1f2b-40aa-bd77-7f7998c3afb2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:59.473847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:59.474888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.475401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.475884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.476996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:59.479116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0ad952c9-ef3f-4d99-861c-c9ef522c8663 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:59.495190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:59.497340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:07:59.499262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/930] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Jul 29 05:07:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:59.500999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071eaa55-9dfe-498e-bcd1-64a31ba1350f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.500999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071eaa55-9dfe-498e-bcd1-64a31ba1350f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.500999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071eaa55-9dfe-498e-bcd1-64a31ba1350f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.503674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/931] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:59.508212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071eaa55-9dfe-498e-bcd1-64a31ba1350f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:59.535146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:59.536564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.537283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.537958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.539589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:59.540370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:59.542801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501b316-496e-45f4-b229-de31adbff217 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:59.553361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:59.559919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:59.577968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/932] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:07:59.580119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.580594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.581017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.605786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:07:59.607132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:07:59.688493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071eaa55-9dfe-498e-bcd1-64a31ba1350f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:59.715428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/933] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:07:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:07:59.715428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:59.747116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:59.747913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.748359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.748717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.750149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:07:59.750553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:59.752142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347f21-d719-4087-b04c-1f0dbf92fd93 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:59.767278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/934] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:07:59.767278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdeac-a277-49cb-8005-0601acfa85e4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.767278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdeac-a277-49cb-8005-0601acfa85e4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.767278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdeac-a277-49cb-8005-0601acfa85e4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.767278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/935] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:07:59.769545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefdeac-a277-49cb-8005-0601acfa85e4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:59.862648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f465-3c7f-4bce-927a-77257a37306d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:59.889160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:59.897063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:07:59.906083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:59.906754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.907046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.907331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.921170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:59.921170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:59.929138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefdeac-a277-49cb-8005-0601acfa85e4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:07:59.930569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:07:59.932271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/936] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:07:59.934399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:07:59.976544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:07:59.977179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:07:59.977467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:07:59.977747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:07:59.978441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:07:59.979978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:07:59.982689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING keystone.server.flask.common [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 05:08:00.006017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 05:08:00.006017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 05:08:00.006017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [parameters: {'id': 'b05af0deb77640ffa8fc666cdee359bc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 05:08:00.006017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fa2bf2e0-0904-4508-a79f-c33af6250fec None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b05af0deb77640ffa8fc666cdee359bc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b05af0deb77640ffa8fc666cdee359bc. Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 05:08:00.023164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 29 05:08:00.024738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Jul 29 05:08:00.025979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 05:08:00.028012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 05:08:00.030536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 05:08:00.032407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'id': 'b05af0deb77640ffa8fc666cdee359bc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.034290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:00.035836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b05af0deb77640ffa8fc666cdee359bc. Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/937] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Jul 29 05:07:59 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.037500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.038720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.048339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:00.049923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:00.124959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/938] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:00.125656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0bb2a-f98a-4e89-9004-44103bb1ad57 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.125821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.178229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.179049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.179437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.179912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.181089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:00.181603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.184052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.191632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/939] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:07:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:00.197951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.232260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:00.272820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:00.276169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77da9-bd2c-44b7-8d16-26ab833a9f86 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.332285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.332889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588cc57-c270-46c2-8c14-f4ae73230960 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.333364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.333869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.334352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.335757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:00.336327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.339462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.367719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/940] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:08:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:00.369228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.375589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.376535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.376989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.377446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.388792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.391258 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:08:00.405164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:00.407177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/941] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:00.409018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.506135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2051df-5e25-43dc-b7c7-417234bfbabd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.537848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:00.545093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.546442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.546935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.547377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.547968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.549390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:00.559391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:00.561581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:00.563679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/942] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:08:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:00.565357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.581119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff56996-99ed-47c4-9d21-dbc2105d7b48 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.598911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:00.627242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.627915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.628263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.628619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.629484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.631172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.659448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/943] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:08:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:08:00.660575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.724265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007421-62cf-42f0-aa2e-2b577c0214f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.803147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.804268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.804725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.805218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/8d737070d3404633a4f6fdd25caa8964/roles/644cadd7a0a0442493993029a8b2a2b6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.809025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=8d737070d3404633a4f6fdd25caa8964, role_id=644cadd7a0a0442493993029a8b2a2b6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.812108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.843391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/944] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:08:00 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/8d737070d3404633a4f6fdd25caa8964/roles/644cadd7a0a0442493993029a8b2a2b6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:08:00.843391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:00.862162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcda1c-2b1a-4687-bb2c-71c4b22030e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:00.911033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:00.912199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:00.912199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:00.912451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:00.913147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:00.913445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:00.916135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:00.945846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/945] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:00.949437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:01.063039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0c7d-4dcc-4431-b4fe-c51b5b55bc60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:01.135657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:01.136708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:01.137214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:01.137710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:01.138958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:01.140306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:01.142749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:01.153192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/946] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:01.160642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:01.216615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:01.335666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6429ed-9278-4106-85ff-f39c10fb243b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:01.406283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:01.407483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:01.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:01.408533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:01.409740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:01.410297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:01.413254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:01.424700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/947] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:01.434389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:01.638823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5635c3f-1fda-469b-8edf-73b5560250eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:01.667867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:01.698670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:01.699306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:01.699630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:01.699910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:01.700757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] RBAC: Adding query filter params (interface=public, service_id=5475c0c3f74e4dae9905bef1d32d4a23, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:01.701078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:01.702750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0419f-10d8-4b17-82de-032c87649177 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:01.709425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/948] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:08:01 2022] GET /identity/v3/endpoints?service_id=5475c0c3f74e4dae9905bef1d32d4a23&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:01.719831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:01.899812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:01.918625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1c451-d00d-444f-b8a0-5e0d2540565c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:01.988845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aadfb-a53f-4d87-8eff-f2d4d7e98f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:01.989959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aadfb-a53f-4d87-8eff-f2d4d7e98f74 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:01.990478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aadfb-a53f-4d87-8eff-f2d4d7e98f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:01.990990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aadfb-a53f-4d87-8eff-f2d4d7e98f74 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:01.992320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aadfb-a53f-4d87-8eff-f2d4d7e98f74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:01.995004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aadfb-a53f-4d87-8eff-f2d4d7e98f74 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:02.008996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/949] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:08:01 2022] GET /identity/v3/services => generated 1497 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:08:03.315839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f8c95-bd8e-4cde-9a7e-08d303832370 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:03.316817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f8c95-bd8e-4cde-9a7e-08d303832370 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:03.316817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f8c95-bd8e-4cde-9a7e-08d303832370 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:03.318150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/950] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:03.324010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:03.326163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:03.326163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:03.346816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:03.348085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:03.480188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/951] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:03.503578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f8c95-bd8e-4cde-9a7e-08d303832370 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:03.628238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f8c95-bd8e-4cde-9a7e-08d303832370 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:03.668358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:03.669045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:03.669327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:03.669573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:03.684075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:03.689351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:03.693056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:03.700137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/952] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 05:08:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:03.703565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:03.878957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:03.913124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c511fe-1380-4b34-95ad-7426afe72800 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:03.992229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:03.993344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:03.993860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:03.994352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:03.995762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:03.996309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:04.000244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1556d-18fd-493f-8f6f-600a2d914071 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:04.018071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/953] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:08:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:04.024870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:04.075604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:04.199310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ab04a-1cb4-484a-aab3-7956f96a832f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:04.271570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:04.272583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.273042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.273522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.292938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbacd3e-97b3-4e57-bccf-1a376dea60d6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.293532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbacd3e-97b3-4e57-bccf-1a376dea60d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.293726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:04.294098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbacd3e-97b3-4e57-bccf-1a376dea60d6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.296001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:04.297610 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/954] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:04.301043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818c249-2a57-4968-9d22-fe78498ceada None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.301385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818c249-2a57-4968-9d22-fe78498ceada None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.301633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818c249-2a57-4968-9d22-fe78498ceada None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.303195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/955] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:04.306558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.307062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.307605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d6a4c398-0b8d-4e55-b256-41d44562c85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:04.318055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:04.320416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:04.322170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:04.322170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:04.322170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:04.322170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:04.322170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/956] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:04.322641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.322641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.323733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.340504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:04.343040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:04.343996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:04.345207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:04.468613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/957] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:04.468874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/958] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:04.469012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:04.507898 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:04.536773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:04.595547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:04.634211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d24c4-a2d8-402b-bd48-3f96f0984945 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:04.684353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04872de-fbc2-4dc5-a98e-aaa7453ecc4c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:04.696330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:04.697152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.697565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.697974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.698827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:04.699298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:04.701728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:04.724745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/959] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:04.724745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:04.760662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:04.761991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.762496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.763002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.764292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:04.766958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:04.785694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:04.787001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:04.788063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/960] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:04.789613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:04.880515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc660-f3ed-40b6-b78d-f2cc762a917d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:04.926328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:04.927327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:04.927824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:04.928291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:04.929314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:04.929460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:04.931608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:04.935985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING keystone.server.flask.common [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 05:08:04.946457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:04.950910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 05:08:04.950910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 05:08:04.950910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [parameters: {'user_id': '53f67c39537e46b2b354a7c556ed0ee0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 05:08:04.950910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 05:08:04.953311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04555f-b409-4a47-8db1-03d5664aee9b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 29 05:08:04.962360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 05:08:04.964036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 05:08:04.965331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 05:08:04.966328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 05:08:04.967773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 05:08:04.968624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'user_id': '53f67c39537e46b2b354a7c556ed0ee0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:04.969647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 29 05:08:04.970569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 29 05:08:04.971866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 29 05:08:04.971866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 29 05:08:04.971866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 05:08:04.971866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 05:08:04.971866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 05:08:04.971866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:04.974528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/961] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 05:08:04 2022] POST /identity/v3/users => generated 160 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:04.976569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.001430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.002105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.002395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.002670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.015288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.018115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e6fb6ec2-5459-40a4-bf37-e7dc720390b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:08:05.040418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:05.041567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:05.042531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/962] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:08:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:05.044574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7703d7-3fbd-49cf-b4fe-72bbfb926ec4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.045091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7703d7-3fbd-49cf-b4fe-72bbfb926ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.046079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7703d7-3fbd-49cf-b4fe-72bbfb926ec4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.053705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/963] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:05.053705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7703d7-3fbd-49cf-b4fe-72bbfb926ec4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.120926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c788b6-eb9c-4eca-931a-75cbbf18c48f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.169182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.170004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.171019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.171483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.172583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:05.173090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.175150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.191033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/964] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:05.191033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.210841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7703d7-3fbd-49cf-b4fe-72bbfb926ec4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.250984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.251578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.251865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.252146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.260353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.261836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.275316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-48b9f577-f4c8-4aa4-ae57-bfa4fb2afae9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:08:05.275879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:05.277721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:05.279825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/965] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:08:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:05.282116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.282577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.282833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.295747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:05.313782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:05.315856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:05.326305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73f512-5904-4066-8930-76ddff104e2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.362217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.362980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.363269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.363438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.364239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:05.364550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.366018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.390985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/966] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 05:08:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:05.392134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.424675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/967] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:05.426314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.537210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad300c-62c6-458a-ad94-dadf70e760be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.563220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:05.577655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.578308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.578596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.578880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.579610 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:05.579924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.581357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.586521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/968] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:05.587402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.603393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701e864-5d16-4f7b-946e-a5ef973db6ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.659218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.660369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.660869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.661355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.662907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:05.663504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.666365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.670845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:05.743425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/969] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 05:08:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:05.743425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:05.780611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b0e2-aae4-4677-94a5-5eeb698aab21 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.847259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.847846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.848137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.848421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.923233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b686b-35be-4ab4-be96-d4c3d53cae91 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:05.964112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:05.973289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:05.974366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:05.974817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:05.975248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:05.976100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.977930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:05.989363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:05.990615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:05.991527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:05.993366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:05.993894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/970] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:08:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:05.996057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:06.005339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:06.132370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:06.219547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/971] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:05 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:06.222736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb861f9f-9886-4d1a-8be1-363bdc380f3d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:06.225626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:06.298066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:06.299149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:06.299704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:06.300207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:06.301786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:06.302336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:06.304935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:06.316399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/972] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:06.326210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:06.347639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:06.369807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352321bf-2c7e-4708-89d4-9d471c033574 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:06.418618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:06.419731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:06.420702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:06.421136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:06.435301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:06.437213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:06.456928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:06.458971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:06.459984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:06.459984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:06.459984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:06.459984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:06.459984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/973] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:06.468203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:06.513301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f2a2-5235-4c9c-9c89-fd8ff78a23ec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:06.542988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:06.563037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:06.564216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:06.564647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:06.565056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:06.566001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:06.566496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:06.568084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:06.574741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/974] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:06.584687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:06.647636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b874b9-44d9-4ea7-9c19-f4613b274227 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:06.692241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:06.692945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:06.693242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:06.693526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:06.694236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:06.694544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:06.696013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:06.706483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:06.713040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/975] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:06.722709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:06.761826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecafa6-ec31-4aee-ab69-b34b399a3224 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:06.817213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:06.817882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:06.818167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:06.818445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:06.819173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:06.820637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:06.848501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/976] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:08:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:08:06.939054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943ce5-4838-42dc-9b14-57ebe8ef05f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:07.016232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:07.017141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:07.017609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:07.018043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:07.030080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:07.032569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:08:07.054328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:07.055934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:07.057435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/977] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:07.063603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:07.125903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:07.214029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c0acb-3fb4-448f-877e-5ecfe2fdae8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:07.273719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:07.274710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:07.275210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:07.275720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:07.277032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:07.277534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:07.280649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92012d5a-fba3-427d-92ff-508eb84b6bc1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:07.304514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/978] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:08:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:07.313629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:07.422068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:07.509024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483353a8-79fe-4c6c-a74f-732991a2b576 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:07.554685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:07.555259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:07.555581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:07.555864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:07.556681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] RBAC: Adding query filter params (role.id=644cadd7a0a0442493993029a8b2a2b6, scope.project.id=cd00b15fba514550924237d11784db71, user.id=8d737070d3404633a4f6fdd25caa8964) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:07.556998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:07.558505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5873d-6f08-4a41-9237-e5f0dd4e11c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:07.567448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/979] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:07 2022] GET /identity/v3/role_assignments?user.id=8d737070d3404633a4f6fdd25caa8964&scope.project.id=cd00b15fba514550924237d11784db71&role.id=644cadd7a0a0442493993029a8b2a2b6 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:07.885360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700d52-c47a-4342-8e0e-8bb8d1bc55ee None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:07.885891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700d52-c47a-4342-8e0e-8bb8d1bc55ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:07.885891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700d52-c47a-4342-8e0e-8bb8d1bc55ee None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:07.887220 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/980] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:07.893730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb993ea-35aa-44c5-b501-9234e5406cdf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:07.894028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb993ea-35aa-44c5-b501-9234e5406cdf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:07.894315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb993ea-35aa-44c5-b501-9234e5406cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:07.914772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5bb993ea-35aa-44c5-b501-9234e5406cdf None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:07.916384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bb993ea-35aa-44c5-b501-9234e5406cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:07.999242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/981] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:08.023169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700d52-c47a-4342-8e0e-8bb8d1bc55ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:08.164882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:08.171440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700d52-c47a-4342-8e0e-8bb8d1bc55ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:08.219014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c500d3-a913-4dc9-93cc-4fdfb7783850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:08.219700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c500d3-a913-4dc9-93cc-4fdfb7783850 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:08.219988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c500d3-a913-4dc9-93cc-4fdfb7783850 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:08.220270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c500d3-a913-4dc9-93cc-4fdfb7783850 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:08.220993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c500d3-a913-4dc9-93cc-4fdfb7783850 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:08.222443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c500d3-a913-4dc9-93cc-4fdfb7783850 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:08.246487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/982] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:08:08 2022] POST /identity/v3/services => generated 242 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:08:09.142141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1765a-5f6d-44e3-a341-50043046c47c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.142999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1765a-5f6d-44e3-a341-50043046c47c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.142999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1765a-5f6d-44e3-a341-50043046c47c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.145296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/983] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:09.152534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.153276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.153443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.179805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:09.181003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:09.271408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/984] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:09.296577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1765a-5f6d-44e3-a341-50043046c47c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:09.353423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:09.409715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1765a-5f6d-44e3-a341-50043046c47c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:09.452552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:09.453686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.454167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.454639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.455801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:09.457802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:09.469649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:09.472352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:09.474473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/985] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:08:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:09.483214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:09.599092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:09.621863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc408a1-93e3-4ec8-8720-7f5bf5dc4f10 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:09.665058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:09.665610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.665894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.666164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.666951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:09.667253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:09.668827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271358f7-90d5-4e5a-97e4-dd7fc259fd68 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:09.676325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/986] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:09.682908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:09.815544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36686036-a9f9-4b0e-8edb-ff6df136895b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:09.833440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:09.870619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:09.871804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.872068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2597f-bbd3-4920-8d8e-8e237efcc173 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.872341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.872528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2597f-bbd3-4920-8d8e-8e237efcc173 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.872687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2597f-bbd3-4920-8d8e-8e237efcc173 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.873053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.874393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:09.874815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:09.875679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/987] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:09.876438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:09.881161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:09.881449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:09.881724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:09.884037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/988] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:09.897160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:09.907982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:09.909230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:10.015208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/989] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:10.039098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100da6c-ef2e-4cbb-a384-b608089846b7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:10.053186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:10.079919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:10.080563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.080848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.081133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.081916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] RBAC: Adding query filter params (interface=admin, service_id=5475c0c3f74e4dae9905bef1d32d4a23, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:10.082231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:10.083679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:10.089938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/990] 10.222.0.49 () {56 vars in 1249 bytes} [Fri Jul 29 05:08:09 2022] GET /identity/v3/endpoints?service_id=5475c0c3f74e4dae9905bef1d32d4a23&interface=admin®ion_id=RegionOne => generated 193 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:10.098923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:10.187555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:10.196366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:10.240674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1534c-1d60-4e52-8b74-f5ddd6cef79c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:10.282198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde535d5-80ee-46f0-ac2e-8ab143ec1ffe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:10.308841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:10.310483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.311813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.312382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.313549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:10.316186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:10.325555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da837c-4c84-4c6d-836b-602567088da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:10.327105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da837c-4c84-4c6d-836b-602567088da5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.327105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da837c-4c84-4c6d-836b-602567088da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.327597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da837c-4c84-4c6d-836b-602567088da5 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.327941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da837c-4c84-4c6d-836b-602567088da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:10.329361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da837c-4c84-4c6d-836b-602567088da5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:10.333730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:10.335712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:10.341285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/991] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Jul 29 05:08:10 2022] GET /identity/v3/services/image => generated 86 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:10.344755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/992] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:08:10 2022] GET /identity/v3/services => generated 1727 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:08:10.347025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:10.563167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403212a-cb03-4fed-969d-acf3672bbb15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:10.621803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc1cbf-2400-4c79-8f13-161ed186ecde None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.622155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc1cbf-2400-4c79-8f13-161ed186ecde None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.622386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc1cbf-2400-4c79-8f13-161ed186ecde None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.624730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/993] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:10.625287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:10.626445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.626953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.627491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.628881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:10.629440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:10.630070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.630381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.630671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.632067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:10.647501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/994] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:10.648641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:10.657399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:10.659178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:10.662343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:10.760792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/995] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:10.784747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c0241-aefc-4de1-82f3-11466668d032 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:10.785569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:10.837645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:10.838411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.838759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:10.838942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.839312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.840851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:10.841041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:10.843967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:10.851382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/996] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:10.927603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722ddd-c6d9-4578-b6ab-58661574cfc8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:10.970851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:10.971579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:10.971870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:10.972146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:10.972797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:10.974238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:10.985122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:10.986934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:10.989045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:10.991822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/997] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:10.993807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:11.118624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:11.174619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849fa93-d1f2-4621-a070-497b90b600bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:11.222199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:11.223171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:11.223171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:11.223469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:11.224101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:11.224482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:11.225916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:11.234508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/998] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:11.238416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:11.360711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:11.366157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a3a2d-7521-40a5-95ec-f025a66d815f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:11.408953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:11.409720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:11.410095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:11.410462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:11.411272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:11.411716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:11.413386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:11.426643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/999] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:11.428579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:11.563027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a5ebb-18ba-46e6-b68d-12905f5aad80 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:11.605635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:11.606533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:11.606973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:11.607511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:11.608495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] RBAC: Adding query filter params (interface=public, service_id=b6ecab0d283e45a094d7fc4da753debb, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:11.609019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:11.610700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7214719-8691-481f-bd9a-3d92c6971a67 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:11.614196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:11.624206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/1000] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:08:11 2022] GET /identity/v3/endpoints?service_id=b6ecab0d283e45a094d7fc4da753debb&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:11.626776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:11.754344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5db94a-0dbe-4491-a249-288eb8680c82 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:11.793931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0b76-9727-46e8-90b4-8c6a342744fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:11.794567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0b76-9727-46e8-90b4-8c6a342744fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:11.794847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0b76-9727-46e8-90b4-8c6a342744fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:11.795771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0b76-9727-46e8-90b4-8c6a342744fd None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:11.796377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0b76-9727-46e8-90b4-8c6a342744fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:11.797785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0b76-9727-46e8-90b4-8c6a342744fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:11.805552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/1001] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:08:11 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:08:13.024276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f6f07-d3fe-44c5-bb9f-bfcdfa37f6ed None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.024856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f6f07-d3fe-44c5-bb9f-bfcdfa37f6ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.025269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f6f07-d3fe-44c5-bb9f-bfcdfa37f6ed None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.027898 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/1002] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:13.034182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09291-28eb-4d5f-a0d8-03878ed74742 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.034841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09291-28eb-4d5f-a0d8-03878ed74742 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.034841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09291-28eb-4d5f-a0d8-03878ed74742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.055446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7de09291-28eb-4d5f-a0d8-03878ed74742 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:13.056688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7de09291-28eb-4d5f-a0d8-03878ed74742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:13.083293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:13.145381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/1003] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:13.173603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f6f07-d3fe-44c5-bb9f-bfcdfa37f6ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:13.323289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f6f07-d3fe-44c5-bb9f-bfcdfa37f6ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:13.366741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3512a5-4e39-483f-adc0-8444d51bb21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:13.367544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3512a5-4e39-483f-adc0-8444d51bb21c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.367864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3512a5-4e39-483f-adc0-8444d51bb21c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.368114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3512a5-4e39-483f-adc0-8444d51bb21c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.368802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3512a5-4e39-483f-adc0-8444d51bb21c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:13.370219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3512a5-4e39-483f-adc0-8444d51bb21c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:13.392844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/1004] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:08:13 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:08:13.885628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a8448-32be-4abe-a7cf-c5197121e299 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.886220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a8448-32be-4abe-a7cf-c5197121e299 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.886654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a8448-32be-4abe-a7cf-c5197121e299 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.890147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/1005] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:13.897967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.898582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc75635-7046-402d-acf3-855a56cb230d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.898750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.898931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc75635-7046-402d-acf3-855a56cb230d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.899068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.899387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc75635-7046-402d-acf3-855a56cb230d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.902677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1006] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:13.911738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:13.912256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:13.912743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:13.933725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:13.936569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:13.944821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:13.947165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:14.045738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/1007] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:13 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:14.056937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:14.079257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1008] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:13 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:14.087469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.112191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.228366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:14.258417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facacb67-3a56-4046-b07b-248c1f585e3e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:14.293339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72013-b962-4b77-be53-9b0eb0f98253 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:14.323665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:14.324708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:14.325288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:14.325584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:14.327629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:14.329190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:14.345081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:14.347168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:14.349323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/1009] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:14.353307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.363836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:14.364676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:14.365081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:14.365500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:14.366284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:14.367929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:14.380613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:14.382838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 05:08:14.384852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:14.387592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1010] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 05:08:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:14.391631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.467245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:14.542772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d85d3-6086-419d-b3e5-ed6518b8fd5c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:14.551919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae97045-d140-4edd-9325-86e487a8748b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:14.603899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:14.604951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:14.605381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:14.605934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:14.606862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:14.607617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:14.609277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:14.621511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/1011] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:14.623732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.625066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:14.625908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:14.626353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:14.626748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:14.627623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:14.628067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:14.629861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:14.639842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1012] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:14.644101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.649491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:14.778912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570682a-1649-45c2-b275-27d18a020dd6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:14.789930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d462454-59d1-499c-b599-eec14a7baad5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:14.813770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:14.832417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:14.833061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:14.833344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:14.833620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:14.834305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:14.834611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:14.836044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:14.836336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:14.837678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:14.838447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:14.838956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:14.840176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:14.840729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:14.841845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1013] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 05:08:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:14.844418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:14.854766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/1014] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 05:08:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:14.855329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.856449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:14.970638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:15.001952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412df27-821f-4d25-bcfa-339bf69798f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:15.002671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fd185-9c6a-4016-8480-b71232055dc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:15.052304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcee2aa-a385-458b-bfab-ded9e857c64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:15.053622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcee2aa-a385-458b-bfab-ded9e857c64b None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:15.054282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcee2aa-a385-458b-bfab-ded9e857c64b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:15.055156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcee2aa-a385-458b-bfab-ded9e857c64b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:15.056629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcee2aa-a385-458b-bfab-ded9e857c64b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:15.056935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2fca6-5fa5-4cdc-9245-7b90db2ee5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:15.057932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2fca6-5fa5-4cdc-9245-7b90db2ee5ee None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:15.058413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2fca6-5fa5-4cdc-9245-7b90db2ee5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:15.058889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2fca6-5fa5-4cdc-9245-7b90db2ee5ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:15.060102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2fca6-5fa5-4cdc-9245-7b90db2ee5ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:15.061609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcee2aa-a385-458b-bfab-ded9e857c64b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:15.062822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2fca6-5fa5-4cdc-9245-7b90db2ee5ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:15.083943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1015] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:08:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:08:15.101564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/1016] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:08:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:08:16.008423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260eb5e-2abb-41f6-87ce-fdaba9a6b7be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:16.008882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260eb5e-2abb-41f6-87ce-fdaba9a6b7be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:16.009338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260eb5e-2abb-41f6-87ce-fdaba9a6b7be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:16.011970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1017] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:16.019551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:16.020069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:16.020578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:16.052040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:16.053286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:16.162759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/1018] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:16.186258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260eb5e-2abb-41f6-87ce-fdaba9a6b7be None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:16.238719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:16.311681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260eb5e-2abb-41f6-87ce-fdaba9a6b7be None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:16.356311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:16.358022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:16.358614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:16.359111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:16.360026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:16.361674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:16.371987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:16.373883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:16.374907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:16.382831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1019] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Jul 29 05:08:16 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:16.384650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:16.489776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:16.555759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec739f-8b50-47cc-9706-6a005341aabc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:16.599908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:16.600503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:16.600788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:16.601066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:16.601781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:16.602094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:16.603575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:16.610468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/1020] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:16.616431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:16.740414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d90120-9dda-445b-88fa-65136335ed0a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:16.752144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:16.786672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:16.787304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:16.787643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:16.787930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:16.788591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:16.788901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:16.790323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:16.797584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1021] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:16.803644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:16.970986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873438a-4c55-4cd0-bc17-227a24cc3eaa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:17.011514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:17.038352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:17.039686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.040308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.040988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.042476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] RBAC: Adding query filter params (interface=public, service_id=92d00307d67447b28f04ad1dff8abeeb, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:17.043273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:17.046620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac86cd-e1fc-46a4-8f11-e349e666cbd4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:17.055488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/1022] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:08:16 2022] GET /identity/v3/endpoints?service_id=92d00307d67447b28f04ad1dff8abeeb&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:17.064464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:17.252232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cac0c-c958-4689-8b82-7b7d77fbf3c4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.253035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cac0c-c958-4689-8b82-7b7d77fbf3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.253035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cac0c-c958-4689-8b82-7b7d77fbf3c4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.254425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/1023] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:17.260075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac28d7-74b7-4792-bce2-83a33b458939 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.260342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac28d7-74b7-4792-bce2-83a33b458939 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.260876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac28d7-74b7-4792-bce2-83a33b458939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.272540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba2293-6b69-4184-a754-e2d502d611c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:17.281264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5aac28d7-74b7-4792-bce2-83a33b458939 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:17.282495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5aac28d7-74b7-4792-bce2-83a33b458939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:17.335016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:17.336090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.336593 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.337069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.338283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:17.340804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:17.363397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1024] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:08:17 2022] GET /identity/v3/services => generated 1957 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:08:17.406606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/1025] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:17.443569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:17.513407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:17.626906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138a6b-ce88-4668-b6ca-118a3ed0721e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:17.640722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32983a97-f08c-41c3-9280-f7d17e830c5a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.641312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32983a97-f08c-41c3-9280-f7d17e830c5a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.641854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32983a97-f08c-41c3-9280-f7d17e830c5a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.644787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/1026] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:17.652979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.658080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.658611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.680336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:17.680999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.681272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.681547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.682776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:17.684285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:17.686808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1027] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:17.688456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:17.689858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:17.692804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:17.787704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/1028] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:17.811101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:17.823286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d76e6-3170-457b-863f-b3c1c17f191d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:17.825170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:17.869289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:17.869951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:17.870226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:17.870508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:17.871222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:17.873014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:17.882501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:17.918872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1029] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:08:17 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:08:17.996889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1846-b3bf-40fc-ae0c-62320c060b61 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:18.072089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:18.072606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:18.073118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:18.073631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:18.092418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:18.097635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:18.098708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:18.100469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1030] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 05:08:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:18.110421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:18.248520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dabd4-64a7-4875-a8e3-9af611b3836a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:18.292350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:18.293330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:18.293330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:18.293578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:18.294269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:18.294612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:18.296072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:18.306773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1031] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:08:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:18.313901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:18.496849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc4744-33a5-4233-a09c-9c147eab5ec8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:18.501713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:18.559809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:18.561008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:18.561507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:18.561969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:18.582237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:18.585084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:18.613370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:18.614345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:18.615282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:18.615282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:18.615282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:18.615282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:18.620496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1032] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:18.622574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:18.645822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:18.744431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026a02-effb-4d1f-8ace-2af68ff9d76a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:18.782948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:18.783487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:18.783779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:18.784060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:18.784763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:18.785070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:18.786468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:18.800298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1033] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:18.806699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:18.992223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee93cb-50fb-4b6e-86bc-38e3384ac93f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:19.058173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:19.059171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:19.059714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:19.061291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:19.076666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:19.079387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:08:19.102619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:19.103956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:19.110427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1034] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:08:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:19.113330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:19.269478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:19.313654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b00d25-fe0c-4933-a4fa-92fe39a031ba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:19.377578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:19.378635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:19.379134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:19.379686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:19.381182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:19.381739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:19.384383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:19.418571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1035] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:08:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:19.428108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:19.465021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:19.634824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63771c5-ec9d-47f6-88f5-119ab67da71d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:19.702222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:19.703329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:19.704029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:19.704547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:19.837326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:19.840146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:19.876327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:20.055383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1036] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:19 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:20.058841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:20.259218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154d0a-fecc-4fd4-9895-6664d13c8158 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:20.323455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:20.324500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.325141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.326268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.329991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:20.344802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:20.346289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ba0e8ffc-e8e7-45c3-9e19-0758842ac2d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:20.364926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:20.366261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:20.368606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:20.368606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:20.368606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:20.368606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:20.370952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1037] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:20.372901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:20.437532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29449-851b-43c2-88c6-59c92a485688 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.437980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29449-851b-43c2-88c6-59c92a485688 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.438453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29449-851b-43c2-88c6-59c92a485688 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.440271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1038] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:20.446225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f7f7a-97fd-412d-a20c-73e2e48dee47 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.446516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f7f7a-97fd-412d-a20c-73e2e48dee47 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.446789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f7f7a-97fd-412d-a20c-73e2e48dee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.472419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-814f7f7a-97fd-412d-a20c-73e2e48dee47 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:20.474335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-814f7f7a-97fd-412d-a20c-73e2e48dee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:20.541485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6a1c9-e00d-47a6-96ab-5b1a0bd5fc6f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:20.554546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1039] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:20.555915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee703ff6-db99-4631-b282-7ed74d2c1695 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.556068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee703ff6-db99-4631-b282-7ed74d2c1695 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.556284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee703ff6-db99-4631-b282-7ed74d2c1695 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.557638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1040] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:20.562902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:20.564961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.565186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.565709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.586089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:20.587225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:20.593803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:20.594842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.595382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.595899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.597785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:20.598373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:20.600739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:20.619609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1041] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:20.621486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:20.667332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1042] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:20.668232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:20.721380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:20.745907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1106787-a1f8-4461-b1b9-023f450cb5dc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:20.784715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc2a0c-7e6f-4515-b155-69ebe3b341de None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:20.787910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:20.788755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.789140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.789578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.790639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:20.793082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:20.811576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:20.813710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:20.815820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1043] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Jul 29 05:08:20 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:20.817581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:20.853926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:20.855003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:20.855489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:20.855922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:20.870094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:20.873211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:20.875653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:08:20.896164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:20.897425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:20.898977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1044] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:20.900008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:20.969296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234337b7-59f0-4317-a656-4609e905a50d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.021175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.021987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.022363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.022770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.024360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.025929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.028091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1045] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:21.029670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:21.052976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0e6ea-c109-43ef-bd8b-bb04f807d430 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.065339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:21.116200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.117193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.117634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.118094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.119373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:21.119925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.122872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.139971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1046] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:21.139971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:21.148782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd1c36-5950-4721-8c90-b26722687c98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.159960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:21.189956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.190601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.190887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.191167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.191967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:21.192284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.193763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.207955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1047] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:21.208712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:21.299645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61768d-5b86-435e-b566-e4182c0fee1f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.342363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e394205-127a-4e19-9000-cb235d88ba1d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.345424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.346216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.346378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.346638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.347295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.348884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.375540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1048] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:08:21 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:08:21.376296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:21.405220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.405875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.406168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.406438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.407154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:21.407563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.409047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.415048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1049] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:21.416595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:21.423025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:21.485335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:21.516765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b16f4-49f5-47e3-9e33-7473261bdd8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.561835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.562852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.563637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.563837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.565252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] RBAC: Adding query filter params (role.id=644cadd7a0a0442493993029a8b2a2b6, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1d28b235f4434aa086cdfc0fc81c7551) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:21.565429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.567625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6508b-83c6-424b-a475-8725199197fe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.580251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1050] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:21 2022] GET /identity/v3/role_assignments?user.id=1d28b235f4434aa086cdfc0fc81c7551&scope.project.id=cd00b15fba514550924237d11784db71&role.id=644cadd7a0a0442493993029a8b2a2b6 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:21.594812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05d889-0335-4f25-b65e-20f5d8df3f14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:21.659508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b3080-f04f-4cfd-9320-0f1dbb5e7185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:21.660585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b3080-f04f-4cfd-9320-0f1dbb5e7185 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:21.661061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b3080-f04f-4cfd-9320-0f1dbb5e7185 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:21.661524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b3080-f04f-4cfd-9320-0f1dbb5e7185 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:21.662827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b3080-f04f-4cfd-9320-0f1dbb5e7185 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:21.665515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b3080-f04f-4cfd-9320-0f1dbb5e7185 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:21.697591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1051] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:08:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:08:23.727751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a11c-15b8-4dbc-a9a1-de64f74e0e7a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:23.728421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a11c-15b8-4dbc-a9a1-de64f74e0e7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:23.728421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a11c-15b8-4dbc-a9a1-de64f74e0e7a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:23.731393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1052] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:23.737885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56857cb-e091-4d11-8a94-cbc155e16bc1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:23.738429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56857cb-e091-4d11-8a94-cbc155e16bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:23.738642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56857cb-e091-4d11-8a94-cbc155e16bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:23.760590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e56857cb-e091-4d11-8a94-cbc155e16bc1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:23.761820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e56857cb-e091-4d11-8a94-cbc155e16bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:23.867414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1053] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:23.888689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3422a-2d1e-4687-9d96-b631d2c397f8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:23.888989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3422a-2d1e-4687-9d96-b631d2c397f8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:23.889264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3422a-2d1e-4687-9d96-b631d2c397f8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:23.890804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1054] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:23.902931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4e14-47c4-48e6-b2dd-e224e4758a53 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:23.903228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4e14-47c4-48e6-b2dd-e224e4758a53 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:23.904161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4e14-47c4-48e6-b2dd-e224e4758a53 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:23.904525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f523-cb83-4a8f-a2f6-c2324fe23e57 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:23.904887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f523-cb83-4a8f-a2f6-c2324fe23e57 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:23.905169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f523-cb83-4a8f-a2f6-c2324fe23e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:23.905722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1055] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:23.906459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c4e14-47c4-48e6-b2dd-e224e4758a53 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:23.924704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6c92f523-cb83-4a8f-a2f6-c2324fe23e57 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:23.926466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c92f523-cb83-4a8f-a2f6-c2324fe23e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:23.937303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.004803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1056] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:24.006110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.006410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.006687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.033268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:24.034606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:24.045965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.062644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c4e14-47c4-48e6-b2dd-e224e4758a53 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.103419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.104098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.104395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.104670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.117675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:24.121235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1057] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:24.122107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:24.123364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:24.124345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:24.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1058] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 05:08:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:24.130590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.243578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.258780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9696f-e4be-4376-93a6-e4e369e06d34 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.285854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3741256-5fc7-409c-a59b-1d269f67256c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.300608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.301255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.301533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.301799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.302531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:24.302837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:24.304310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:24.313730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1059] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:24.314628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.331657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.332316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.332599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.332874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.336374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.414271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:24.415807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:24.457654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8453bb-778e-44df-a2c5-f213ac144953 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.512086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.512653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.513948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.513948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.522999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.526141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:24.532723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:24.534539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1060] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:24.536301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.568082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1061] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:24.569247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.628953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.676054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e6d0b-3373-4ec6-ab31-8b81d472b1d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.707636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e404d2-7a9e-440d-88d8-d5fbe63faa1e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.735303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.736366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.736821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.737274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.757744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:24.759467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.760122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.760409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.760622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:24.760759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.761409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:24.761720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:24.763913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:24.773264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1062] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:24.774146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:24.789756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:24.791106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:24.792436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:24.792436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:24.792436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:24.792436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:24.792436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1063] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:24.793853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:24.823248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.911485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffa07f-cba2-404d-8e2a-6b68045ab1ba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.940230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968814d2-2a59-4e12-96b9-79c7fc8bba31 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:24.955411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.956100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.956394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.956681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.970704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:24.972300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:24.972838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:24.988204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:24.988871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:24.989157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:24.989441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] PATH_INFO: `/v3/users/e87c3945c812427d8576b63fa1acd4e8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:24.990765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:24.992757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:24.994554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:24.994554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:24.994554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:24.994554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:24.994554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1064] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:24.994554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.013901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] RBAC: Authorizing `identity:get_user(user_id=e87c3945c812427d8576b63fa1acd4e8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.015836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.017772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1065] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/users/e87c3945c812427d8576b63fa1acd4e8 => generated 285 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:25.018852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.129136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:25.143875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82415e51-4fcc-4408-b6e6-4fdcf634f4f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.163473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac253f6d-04cc-453e-b6cb-5a72b729b070 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.186292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.187063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.187500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.187882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.189002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:25.189475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.191045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.209931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1066] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:25.211549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.226095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.226872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.227734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.228107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.229322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:25.229742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.231372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.246055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1067] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:25.247228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.328745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:25.372912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938c68-b7e0-4f59-85a3-be4c803b645e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.438358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.439646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.440278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.440703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.443569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aa85d-4546-426e-a833-c08af597f98e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.478708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:25.513935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.515060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.515581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.516030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.529183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.531259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:08:25.545879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:25.547748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:25.549565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1068] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:25.553904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.581006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.583706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.710231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f581-92af-4ec4-a4c1-9a9560e3ab15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.713413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:25.719487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1069] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:25.726324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.754864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.755545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.755827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.756105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.756906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:25.757212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.758661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.780500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:25.783557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1070] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:25.787881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.836388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcc8ff-074d-45b7-91f8-a4ef1b7f2c45 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.873311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.874086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.874365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.875906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] RBAC: Authorizing `identity:get_project(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.877366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.879085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1071] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:25.885409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:25.912191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28164d-c9cc-46c7-b0c9-2faa90ba1d3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:25.959469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:25.960635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:25.961153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:25.961636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:25.980442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:25.984034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:25.999747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b6a8c-041c-4107-9e7a-64c75570f897 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:26.010205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:26.011236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:26.012213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:26.012213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:26.012213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:26.012213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:26.012441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1072] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:26.021706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:26.021984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:26.043074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:26.043734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:26.044552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:26.044834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:26.045790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6, user.id=e87c3945c812427d8576b63fa1acd4e8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:26.046121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:26.046275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:26.047656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:26.054125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1073] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:25 2022] GET /identity/v3/role_assignments?user.id=e87c3945c812427d8576b63fa1acd4e8&scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:26.202074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f784f1-5c25-4eec-9b1d-d3efa0bd9bc9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:26.249032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:26.249791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:26.250138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:26.250476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:26.251252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:26.251680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:26.253153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:26.267397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1074] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:26.273166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:26.379622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173fc-4f1d-4954-adbd-7b007cb4a4ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:26.416664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:26.417332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:26.417620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:26.417899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:26.426227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:26.427734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:08:26.440052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:26.441453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:26.442518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1075] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:26.447374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:26.530186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:26.563082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e911a-6d7e-468b-b689-295198871b37 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:26.598477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:26.599112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:26.599427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:26.599704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:26.600457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:26.600766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:26.602207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0987b-0666-47cf-9972-a0fe4f81111f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:26.614910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1076] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:26.621193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:26.667305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:26.733102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90dca8-7125-4dfb-820e-35ce171bf4d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:26.769602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fa3ed-28fd-48d1-aeb3-f289283fc875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:26.770294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fa3ed-28fd-48d1-aeb3-f289283fc875 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:26.770580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fa3ed-28fd-48d1-aeb3-f289283fc875 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:26.770861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fa3ed-28fd-48d1-aeb3-f289283fc875 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/1d28b235f4434aa086cdfc0fc81c7551/roles/644cadd7a0a0442493993029a8b2a2b6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:26.773155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fa3ed-28fd-48d1-aeb3-f289283fc875 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=1d28b235f4434aa086cdfc0fc81c7551, role_id=644cadd7a0a0442493993029a8b2a2b6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:26.774619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fa3ed-28fd-48d1-aeb3-f289283fc875 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:26.790107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1077] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:08:26 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/1d28b235f4434aa086cdfc0fc81c7551/roles/644cadd7a0a0442493993029a8b2a2b6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:08:28.836653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa8ccc-d879-4cd4-adc7-5cd556dee9b4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:28.837194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa8ccc-d879-4cd4-adc7-5cd556dee9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:28.837286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa8ccc-d879-4cd4-adc7-5cd556dee9b4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:28.838625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1078] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:28.845632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:28.846142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:28.846237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:28.866537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:28.867794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:28.942888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1079] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:28.966553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa8ccc-d879-4cd4-adc7-5cd556dee9b4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.097596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa8ccc-d879-4cd4-adc7-5cd556dee9b4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.135142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:29.141905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00920d0c-4918-42c8-9a8d-efc0eae58fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.142566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00920d0c-4918-42c8-9a8d-efc0eae58fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.142851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00920d0c-4918-42c8-9a8d-efc0eae58fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.143130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00920d0c-4918-42c8-9a8d-efc0eae58fd4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.156007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00920d0c-4918-42c8-9a8d-efc0eae58fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-00920d0c-4918-42c8-9a8d-efc0eae58fd4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:29.161231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:29.163337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 05:08:29.164692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:29.167699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1080] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 05:08:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:29.169450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.207582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:29.221112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bdef3-9ab2-425f-bda7-540675c2c831 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.221398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bdef3-9ab2-425f-bda7-540675c2c831 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.221681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bdef3-9ab2-425f-bda7-540675c2c831 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.223090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1081] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:29.232266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.232712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.233158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.256401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:29.257762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:29.306249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030034e-d1de-4f0f-b0f8-40038720470d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.347569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1082] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:29.348352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.348920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.349200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.349476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.350211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:29.350518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:29.351989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:29.362269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1083] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:29.368345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.374721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.488054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:29.495517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f88c4-1bb5-4f0f-967e-cfd1836586c8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.502032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c05eb1-48c1-468f-85a5-e7bfb4fdeeb1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.524526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:29.542615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.543553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.544112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.544459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.558244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.558981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:29.559158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.560066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.560409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.561734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:29.574853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:29.579776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:29.580962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:29.582058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:29.582058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:29.582058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:29.582058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:29.582058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1084] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:29.582429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:29.583294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:29.584229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:29.587248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1085] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:29.587526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.588201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.711652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b603c-ad96-436c-b3d3-dca26c2e3850 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.737120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041a361-3372-422f-a1b9-4f9eb2e822c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.753581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.754218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.754507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.754790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.755533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:29.755848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:29.757262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:29.770608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1086] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:29.775658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.776297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.776495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.776601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.776847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.777577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:29.780855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:29.782345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:29.792200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1087] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:29.799157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.816872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:29.818402 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:29.897087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473edc1-a285-4415-ab4f-0c8cd656bcfd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.917069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7d525-5c88-4e83-9762-c411e92d6af1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:29.947338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.948034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.948322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.948610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.957764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:29.958106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:29.958777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:29.959061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:29.959249 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:29.959380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] PATH_INFO: `/v3/users/e87c3945c812427d8576b63fa1acd4e8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:29.960685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] RBAC: Authorizing `identity:get_user(user_id=e87c3945c812427d8576b63fa1acd4e8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:29.964808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:29.964808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1088] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/users/e87c3945c812427d8576b63fa1acd4e8 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:29.970820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:08:29.973003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:29.974589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 05:08:29.976748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:29.979303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1089] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:29.980317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:30.081127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:30.099282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:30.108419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d141-6091-4dad-99ad-79b70b070a77 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:30.114313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5583d-62eb-4e22-9a32-4607039aa5f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:30.156678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:30.157358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:30.157648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:30.157946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:30.159262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] RBAC: Authorizing `identity:get_project(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:30.160832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:30.162701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1090] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6 => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:30.170318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:30.177576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:30.178657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:30.179121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:30.179660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:30.181189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:30.181754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:30.185202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:30.214358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1091] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 05:08:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:30.222202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:30.319529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d009492-ca4b-4f0b-9a6e-f5099ce5308d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:30.353775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d2de5-313c-4beb-a140-f18283d9808c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:30.371701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:30.372695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:30.373152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:30.373436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6/users/e87c3945c812427d8576b63fa1acd4e8/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:30.375618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6, user_id=e87c3945c812427d8576b63fa1acd4e8, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:30.377127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:30.394761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1092] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:08:30 2022] PUT /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6/users/e87c3945c812427d8576b63fa1acd4e8/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:08:30.411020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:30.412089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:30.412546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:30.413110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:30.444140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:30.542630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:30.546070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:30.692395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1093] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:30.698512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:30.713287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:30.851856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200eb37-9f45-4074-8c9b-f337bde5ca22 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:30.913531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:30.914602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:30.915071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:30.915587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:30.934387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:30.936127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:08:30.954101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:30.956135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:30.958150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:08:30.958150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:08:30.958150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:08:30.958150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:30.959668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1094] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:30.962496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:31.040405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:31.100672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa847-4adc-4465-8215-274eec66e09f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:31.141326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:31.142130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:31.142538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:31.142932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:31.143824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:31.144264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:31.145800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:31.160501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1095] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:31.166463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:31.246121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:31.302447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6449c-e6b5-4fd5-80d1-07eddb3d04ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:31.349661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:31.350340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:31.350624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:31.350901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:31.358897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:31.360441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:08:31.373785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:31.374807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:08:31.375750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:08:31.379562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1096] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:31.381832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:31.496149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92897245-e8fe-47a4-83bc-b2f6615f8da6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:31.512162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:31.536423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:31.537055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:31.537336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:31.537609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:31.538366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:31.538675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:31.540161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9414749-691a-46a7-a65e-2021f496506c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:31.554308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1097] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:31.561791 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:31.695580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8047f-2eb3-4e55-9913-ce7a0c60000e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:31.721636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:31.736113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:31.736859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:31.737231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:31.737571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:31.738476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] RBAC: Adding query filter params (role.id=644cadd7a0a0442493993029a8b2a2b6, scope.project.id=cd00b15fba514550924237d11784db71, user.id=1d28b235f4434aa086cdfc0fc81c7551) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:31.738857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:31.740532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee4eb-3ae4-4b23-9b30-9e52047d59c3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:31.748749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1098] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:31 2022] GET /identity/v3/role_assignments?user.id=1d28b235f4434aa086cdfc0fc81c7551&scope.project.id=cd00b15fba514550924237d11784db71&role.id=644cadd7a0a0442493993029a8b2a2b6 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:32.691894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d05b4-3981-45f1-bbed-9cb2dd7ec3a5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:32.692631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d05b4-3981-45f1-bbed-9cb2dd7ec3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:32.692895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d05b4-3981-45f1-bbed-9cb2dd7ec3a5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:32.695579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1099] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:32.705360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:32.705908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:32.706338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:32.734467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:32.735853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:32.850580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1100] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:32.884991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d05b4-3981-45f1-bbed-9cb2dd7ec3a5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:33.005074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d05b4-3981-45f1-bbed-9cb2dd7ec3a5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:33.042974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:33.043800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:33.044321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:33.044812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:33.057416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:33.063527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:33.065449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:33.067167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1101] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:08:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:33.075474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:33.197536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7244a-e507-4a83-a3af-a843db957c13 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:33.218320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:33.240436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:33.241092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:33.241379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:33.241658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:33.242402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:33.242717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:33.244174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:33.254226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1102] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:08:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:33.264313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:33.301542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:33.378448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c6f61-df6c-4e9c-8f98-c469bf017530 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:33.415910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:33.416563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:33.416848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:33.417127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] PATH_INFO: `/v3/users/e87c3945c812427d8576b63fa1acd4e8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:33.418351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] RBAC: Authorizing `identity:get_user(user_id=e87c3945c812427d8576b63fa1acd4e8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:33.421101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:33.422791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1103] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:33 2022] GET /identity/v3/users/e87c3945c812427d8576b63fa1acd4e8 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:33.429107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:33.543643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779efa5-afe9-4148-b108-16a697a2889a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:33.582641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:33.583381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:33.583716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:33.584075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:33.639364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:33.664097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:33.665590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:33.783949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:33.790140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:33.898998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:33.903373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e97acd-2284-4e87-967d-754d3f05ed15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:33.944012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4991f2-a796-47ef-bfe7-5c4611be027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:33.944786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4991f2-a796-47ef-bfe7-5c4611be027f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:33.945012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4991f2-a796-47ef-bfe7-5c4611be027f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:33.945295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4991f2-a796-47ef-bfe7-5c4611be027f None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:33.946581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4991f2-a796-47ef-bfe7-5c4611be027f None admin] RBAC: Authorizing `identity:get_project(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:33.948089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4991f2-a796-47ef-bfe7-5c4611be027f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:33.951012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:33 2022] GET /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:33.957979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:34.104314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:34.148513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c931cf-b310-4ea2-b566-fb32a6718d24 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:34.198759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:34.199307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:34.199901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:34.199901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:34.201700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6, user.id=e87c3945c812427d8576b63fa1acd4e8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:34.202204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:34.203801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77675a-0c32-442c-b08d-2b053f2de513 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:34.210990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:33 2022] GET /identity/v3/role_assignments?user.id=e87c3945c812427d8576b63fa1acd4e8&scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:36.335284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed307b7-213a-4a25-8532-413d7a079d25 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:36.335882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed307b7-213a-4a25-8532-413d7a079d25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:36.335882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed307b7-213a-4a25-8532-413d7a079d25 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:36.337277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1107] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:36.344665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:36.345216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:36.345216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:36.365956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:36.367216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:36.449334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:36.475098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed307b7-213a-4a25-8532-413d7a079d25 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:36.616747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed307b7-213a-4a25-8532-413d7a079d25 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:36.681390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9f911-2efa-4878-ac1c-45da0fcf2263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:36.682533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f911-2efa-4878-ac1c-45da0fcf2263 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:36.683034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f911-2efa-4878-ac1c-45da0fcf2263 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:36.683571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f911-2efa-4878-ac1c-45da0fcf2263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:36.687061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9f911-2efa-4878-ac1c-45da0fcf2263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:36.689818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9f911-2efa-4878-ac1c-45da0fcf2263 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:36.693067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:36.699078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:36.788024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:36.821925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349030b2-88f6-4a61-8e06-4f3e18c65dad None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:36.867202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2bb5c-837d-4515-ae60-769878969a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:36.868354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bb5c-837d-4515-ae60-769878969a83 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:36.868658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bb5c-837d-4515-ae60-769878969a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:36.868942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bb5c-837d-4515-ae60-769878969a83 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:36.869667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2bb5c-837d-4515-ae60-769878969a83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:36.871339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2bb5c-837d-4515-ae60-769878969a83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:36.897049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:08:36 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:08:39.023858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86724bb6-4416-4022-9c9a-1a372353ec8c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:39.024456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86724bb6-4416-4022-9c9a-1a372353ec8c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:39.024456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86724bb6-4416-4022-9c9a-1a372353ec8c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:39.025900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1111] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:39.032714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:39.033289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:39.033289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:39.053920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:39.055255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:39.138351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:39.162107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86724bb6-4416-4022-9c9a-1a372353ec8c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:39.203962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:39.276960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86724bb6-4416-4022-9c9a-1a372353ec8c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:39.318302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:39.319185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:39.319724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:39.320233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] PATH_INFO: `/v3/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:39.322773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] RBAC: Authorizing `identity:get_role(role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:39.325416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:39.328045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:39 2022] GET /identity/v3/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:39.335518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:39.399173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:39.460817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e86c3-0dfd-44b2-ad92-1f05bb11d102 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:39.499835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:39.500501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:39.500790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:39.501070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] PATH_INFO: `/v3/users/88e4a3e266a046e9803aff55e082b42d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:39.515955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] RBAC: Authorizing `identity:get_user(user_id=88e4a3e266a046e9803aff55e082b42d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:39.517437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:39.519180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:39 2022] GET /identity/v3/users/88e4a3e266a046e9803aff55e082b42d => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:39.525401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:39.633997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa359d12-6e6c-4d13-8cd7-64f2ddaed6a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:39.636516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:39.672074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:39.672711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:39.673003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:39.673284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:39.744698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:39.746793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:39.865838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:39.871833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:39.983566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c64ee-f99f-4431-9e29-0ad168f918f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:40.003570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:40.023041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191822d-e59b-4d71-a31b-5d5ca6c1c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:40.023723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191822d-e59b-4d71-a31b-5d5ca6c1c293 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:40.024005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191822d-e59b-4d71-a31b-5d5ca6c1c293 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:40.024283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191822d-e59b-4d71-a31b-5d5ca6c1c293 None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:40.025468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191822d-e59b-4d71-a31b-5d5ca6c1c293 None admin] RBAC: Authorizing `identity:get_project(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:40.026914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191822d-e59b-4d71-a31b-5d5ca6c1c293 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:40.029289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:39 2022] GET /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:40.036465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:40.039636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:40.153799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7915f-b315-4386-960c-c1b2797354a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:40.202443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:40.203095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:40.203476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:40.203760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:40.204574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] RBAC: Adding query filter params (role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6, user.id=88e4a3e266a046e9803aff55e082b42d) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:40.204892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:40.206394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf7a04-5a12-45b3-8893-be5e1b2864a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:40.213020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:40 2022] GET /identity/v3/role_assignments?user.id=88e4a3e266a046e9803aff55e082b42d&scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:42.267606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26f4e-e83f-498e-9827-9bf7e57db85b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:42.268234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26f4e-e83f-498e-9827-9bf7e57db85b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:42.268234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26f4e-e83f-498e-9827-9bf7e57db85b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:42.269563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1118] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:42.275625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:42.276150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:42.276305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:42.295890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:42.297129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:42.365429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:42.372764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:42.396328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda26f4e-e83f-498e-9827-9bf7e57db85b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:42.509207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda26f4e-e83f-498e-9827-9bf7e57db85b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:42.546465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:42.547120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:42.547437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:42.547722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] PATH_INFO: `/v3/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:42.549482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] RBAC: Authorizing `identity:get_role(role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:42.550878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:42.552280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:42 2022] GET /identity/v3/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:42.558571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:42.668729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d97d-de6b-4fe6-b70a-43c3998f9937 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:42.705598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:42.706259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:42.706548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:42.706838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] PATH_INFO: `/v3/users/88e4a3e266a046e9803aff55e082b42d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:42.708062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] RBAC: Authorizing `identity:get_user(user_id=88e4a3e266a046e9803aff55e082b42d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:42.709515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:42.711179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:42 2022] GET /identity/v3/users/88e4a3e266a046e9803aff55e082b42d => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:42.720814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:42.815871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:42.866629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cfee5-78ae-4445-8344-9585669e7c4a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:42.902032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e9adc-30e4-4653-bd66-fa483e6860d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:42.903444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e9adc-30e4-4653-bd66-fa483e6860d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:42.903736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e9adc-30e4-4653-bd66-fa483e6860d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:42.904019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e9adc-30e4-4653-bd66-fa483e6860d6 None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:42.905230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e9adc-30e4-4653-bd66-fa483e6860d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:42.906700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e9adc-30e4-4653-bd66-fa483e6860d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:42.908475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:42 2022] GET /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6 => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:42.914750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:42.985741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:43.028006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73024-ac76-4e71-b2c1-c8491ef23f26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:43.064772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e49ee-d38e-48c4-8f55-66d8f2e80ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:43.065458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e49ee-d38e-48c4-8f55-66d8f2e80ebc None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:43.065742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e49ee-d38e-48c4-8f55-66d8f2e80ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:43.066072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e49ee-d38e-48c4-8f55-66d8f2e80ebc None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6/users/88e4a3e266a046e9803aff55e082b42d/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:43.068234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e49ee-d38e-48c4-8f55-66d8f2e80ebc None admin] RBAC: Authorizing `identity:create_grant(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6, user_id=88e4a3e266a046e9803aff55e082b42d, role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:43.069695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e49ee-d38e-48c4-8f55-66d8f2e80ebc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:43.083911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:08:42 2022] PUT /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6/users/88e4a3e266a046e9803aff55e082b42d/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:08:45.175646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678fa22-71d1-4b5a-8e59-16c251bb35e7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:45.176484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678fa22-71d1-4b5a-8e59-16c251bb35e7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:45.176638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678fa22-71d1-4b5a-8e59-16c251bb35e7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:45.178120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1124] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:45.184612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:45.185225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:45.185350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:45.205057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:45.206297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:45.289922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:45.332587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678fa22-71d1-4b5a-8e59-16c251bb35e7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:45.459818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678fa22-71d1-4b5a-8e59-16c251bb35e7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:45.484576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:45.501787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:45.502467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:45.502758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:45.503040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] PATH_INFO: `/v3/roles/b23e697fe9b246ffabc2ed7667ff7064` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:45.504575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] RBAC: Authorizing `identity:get_role(role_id=b23e697fe9b246ffabc2ed7667ff7064)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:45.505980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:45.507392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:45 2022] GET /identity/v3/roles/b23e697fe9b246ffabc2ed7667ff7064 => generated 223 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:45.517915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:45.604367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:45.659134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076b1d1-e2bc-4a48-addd-3f72de4b7a7c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:45.700806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:45.701485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:45.701770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:45.702049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] PATH_INFO: `/v3/users/88e4a3e266a046e9803aff55e082b42d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:45.703414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] RBAC: Authorizing `identity:get_user(user_id=88e4a3e266a046e9803aff55e082b42d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:45.704940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:45.706686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:45 2022] GET /identity/v3/users/88e4a3e266a046e9803aff55e082b42d => generated 286 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:45.712950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:45.829063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57709481-5a5d-46d2-ac94-1af3aea0ebef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:45.866694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:45.867334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:45.867655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:45.867940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:45.933403 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:45.948076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:45.949706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:46.070375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:08:46.081004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:46.244406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0306cbb-0f39-4141-8850-94099003fb16 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:46.270610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:46.285091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9410f-41db-4e8e-8e5c-c55c9995df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:46.285831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9410f-41db-4e8e-8e5c-c55c9995df02 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:46.286167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9410f-41db-4e8e-8e5c-c55c9995df02 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:46.286495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9410f-41db-4e8e-8e5c-c55c9995df02 None admin] PATH_INFO: `/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:46.287876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9410f-41db-4e8e-8e5c-c55c9995df02 None admin] RBAC: Authorizing `identity:get_project(project_id=d0a6bd588e3a46fdbe48aef0bd6c79d6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:46.289389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9410f-41db-4e8e-8e5c-c55c9995df02 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:46.291162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:46 2022] GET /identity/v3/projects/d0a6bd588e3a46fdbe48aef0bd6c79d6 => generated 311 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:46.298200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:46.396952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:46.411880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c8ec-d48f-49c4-b5b5-c4262198f946 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:46.452082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:46.452733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:46.453024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:46.453306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:46.454127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] RBAC: Adding query filter params (role.id=b23e697fe9b246ffabc2ed7667ff7064, scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6, user.id=88e4a3e266a046e9803aff55e082b42d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:46.454444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:46.456200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20773f7b-0739-4f25-b68c-db3e832ffb73 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:46.463196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:46 2022] GET /identity/v3/role_assignments?user.id=88e4a3e266a046e9803aff55e082b42d&scope.project.id=d0a6bd588e3a46fdbe48aef0bd6c79d6&role.id=b23e697fe9b246ffabc2ed7667ff7064 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:48.582966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc4bc9-e666-4b33-9662-058df55ec8b9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:48.583648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc4bc9-e666-4b33-9662-058df55ec8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:48.583648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc4bc9-e666-4b33-9662-058df55ec8b9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:48.585003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1131] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:48.591332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:48.591905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:48.592091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:48.611505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:48.613127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:48.686796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:08:48.710224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc4bc9-e666-4b33-9662-058df55ec8b9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:48.825142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc4bc9-e666-4b33-9662-058df55ec8b9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:48.863540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d234c-d653-4f08-851f-9f99dcdb2b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:48.864245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d234c-d653-4f08-851f-9f99dcdb2b99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:48.864625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d234c-d653-4f08-851f-9f99dcdb2b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:48.864993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d234c-d653-4f08-851f-9f99dcdb2b99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:48.866421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d234c-d653-4f08-851f-9f99dcdb2b99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:48.868027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d234c-d653-4f08-851f-9f99dcdb2b99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:48.869762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:48.875953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:48.964989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:48.985682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e148-838b-44d7-b916-2ff05b9949d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:49.022246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62ef8-3124-4753-b6de-7963999b42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:49.022886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62ef8-3124-4753-b6de-7963999b42fa None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:49.023172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62ef8-3124-4753-b6de-7963999b42fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:49.023481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62ef8-3124-4753-b6de-7963999b42fa None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:49.024187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62ef8-3124-4753-b6de-7963999b42fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:49.025636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62ef8-3124-4753-b6de-7963999b42fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:49.054798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:08:48 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:08:51.144962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7474-65e5-42ee-b985-8998b94a025a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:51.145553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7474-65e5-42ee-b985-8998b94a025a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:51.145649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7474-65e5-42ee-b985-8998b94a025a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:51.147390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1135] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:08:51.155027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:51.155542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:51.155653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:51.176099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:51.177322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:51.257041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:51.284003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a7474-65e5-42ee-b985-8998b94a025a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:51.362013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:51.407577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a7474-65e5-42ee-b985-8998b94a025a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:51.450398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da187403-5dd9-4589-b915-5af4cc102499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:51.451052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da187403-5dd9-4589-b915-5af4cc102499 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:51.451331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da187403-5dd9-4589-b915-5af4cc102499 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:51.451642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da187403-5dd9-4589-b915-5af4cc102499 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:51.452818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da187403-5dd9-4589-b915-5af4cc102499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:51.454276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da187403-5dd9-4589-b915-5af4cc102499 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:51.455935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:51.462082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:51.531051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:51.573070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c935dbc-8e3d-4121-b56f-e15b901fe339 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:51.609119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773fcb6-9b21-4d4e-80b7-aabc89c3f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:51.609754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773fcb6-9b21-4d4e-80b7-aabc89c3f8e0 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:51.610041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773fcb6-9b21-4d4e-80b7-aabc89c3f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:51.610322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773fcb6-9b21-4d4e-80b7-aabc89c3f8e0 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:51.611035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773fcb6-9b21-4d4e-80b7-aabc89c3f8e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:51.612533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773fcb6-9b21-4d4e-80b7-aabc89c3f8e0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:51.638786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:08:51 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:08:53.775468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc218d-c323-4cea-9f7b-030a2a14ec28 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:53.776261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc218d-c323-4cea-9f7b-030a2a14ec28 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:53.776261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc218d-c323-4cea-9f7b-030a2a14ec28 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:53.777789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1139] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:53.783709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:53.784286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:53.784416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:53.804447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:53.805672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:53.878103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:53.903964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc218d-c323-4cea-9f7b-030a2a14ec28 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:54.063822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc218d-c323-4cea-9f7b-030a2a14ec28 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:54.097276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:54.107256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:54.107983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:54.108268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:54.108544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:54.121593 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:54.126797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:54.128695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:54.130380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:08:54.138862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:54.209451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:54.281519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78662aaa-c120-4d0a-8f82-1518513e9ca6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:54.320364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:54.320906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:54.321187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:54.321461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:54.322183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:54.322487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:54.324460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:54.333376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:54.343032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:54.495133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f9e02-33a3-47d9-b6a5-cc734f047292 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:54.532846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:54.533502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:54.533784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:54.534060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] PATH_INFO: `/v3/users/b8d053796e4e443c849752ecb53fc9f2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:54.550098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] RBAC: Authorizing `identity:get_user(user_id=b8d053796e4e443c849752ecb53fc9f2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:54.551645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:54.553399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:54 2022] GET /identity/v3/users/b8d053796e4e443c849752ecb53fc9f2 => generated 286 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:54.559607 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:54.681605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cd988-42ef-4c38-9f38-c33aad27e00b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:54.696176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:54.720744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:54.721380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:54.721665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:54.721942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:54.795427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:54.796977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:54.909288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:08:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 05:08:54.915802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:54.968856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:55.026285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54d9d0-902a-445f-ab12-66790d06ba53 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:55.066082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602f630-f8be-461c-b632-64def3b09637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:55.066737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602f630-f8be-461c-b632-64def3b09637 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:55.067019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602f630-f8be-461c-b632-64def3b09637 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:55.067294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602f630-f8be-461c-b632-64def3b09637 None admin] PATH_INFO: `/v3/projects/16b9d419847e43258369829423c21fd0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:55.068710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602f630-f8be-461c-b632-64def3b09637 None admin] RBAC: Authorizing `identity:get_project(project_id=16b9d419847e43258369829423c21fd0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:55.070170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602f630-f8be-461c-b632-64def3b09637 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:55.071927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:54 2022] GET /identity/v3/projects/16b9d419847e43258369829423c21fd0 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:55.081812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:55.107860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:55.201465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb2654-9e0f-491a-a589-d203c22bc436 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:55.239387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:55.239941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:55.240223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:55.240498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:55.241294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=16b9d419847e43258369829423c21fd0, user.id=b8d053796e4e443c849752ecb53fc9f2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:55.241608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:55.243066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0b1e2-3e1f-4bb7-820a-0dc1f2073dc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:55.249121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:08:55 2022] GET /identity/v3/role_assignments?user.id=b8d053796e4e443c849752ecb53fc9f2&scope.project.id=16b9d419847e43258369829423c21fd0&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:57.386874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf183b9-9549-4468-9abc-7e9f916cac35 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:57.387522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf183b9-9549-4468-9abc-7e9f916cac35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:57.387522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf183b9-9549-4468-9abc-7e9f916cac35 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:57.388838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1147] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:08:57.394694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:57.395250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:57.395250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:57.414222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:08:57.415483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:08:57.485772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:08:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:08:57.508836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf183b9-9549-4468-9abc-7e9f916cac35 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:57.615650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf183b9-9549-4468-9abc-7e9f916cac35 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:57.635084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:57.656003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:57.656659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:57.656945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:57.657224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:57.668695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:08:57.674127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:08:57.675222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:08:57.676669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:08:57.681634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:57.690607 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:57.788823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bcdbc-1b04-4682-8be6-341c86b97835 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:57.826949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:57.827627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:57.827912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:57.828188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:57.828920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:08:57.829229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:57.830624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:57.840412 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:08:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:57.846702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:57.967099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310834a-b2ba-4faa-99dc-f94742bd1566 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:58.006115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:58.006764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:58.007049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:58.007324 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] PATH_INFO: `/v3/users/b8d053796e4e443c849752ecb53fc9f2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:58.008562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] RBAC: Authorizing `identity:get_user(user_id=b8d053796e4e443c849752ecb53fc9f2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:58.010008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:58.012314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:08:57 2022] GET /identity/v3/users/b8d053796e4e443c849752ecb53fc9f2 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:08:58.021702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:58.107450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:58.143121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65c327-d2ae-47a3-9427-780d9688cbab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:58.179084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bdbf9-1cee-4377-8885-441d1f45df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:58.179781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bdbf9-1cee-4377-8885-441d1f45df00 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:58.180066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bdbf9-1cee-4377-8885-441d1f45df00 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:58.180344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bdbf9-1cee-4377-8885-441d1f45df00 None admin] PATH_INFO: `/v3/projects/16b9d419847e43258369829423c21fd0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:58.181530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bdbf9-1cee-4377-8885-441d1f45df00 None admin] RBAC: Authorizing `identity:get_project(project_id=16b9d419847e43258369829423c21fd0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:58.183335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bdbf9-1cee-4377-8885-441d1f45df00 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:58.185107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:08:58 2022] GET /identity/v3/projects/16b9d419847e43258369829423c21fd0 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:08:58.191651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:08:58.239261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:08:58.299205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948c157-dfdf-401b-95f3-790423b5be60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:08:58.335186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9063e5f-79c5-4028-b0aa-59a6acb287c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:08:58.335907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9063e5f-79c5-4028-b0aa-59a6acb287c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:08:58.336196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9063e5f-79c5-4028-b0aa-59a6acb287c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:08:58.336475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9063e5f-79c5-4028-b0aa-59a6acb287c0 None admin] PATH_INFO: `/v3/projects/16b9d419847e43258369829423c21fd0/users/b8d053796e4e443c849752ecb53fc9f2/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:08:58.338437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9063e5f-79c5-4028-b0aa-59a6acb287c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=16b9d419847e43258369829423c21fd0, user_id=b8d053796e4e443c849752ecb53fc9f2, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:08:58.339965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9063e5f-79c5-4028-b0aa-59a6acb287c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:08:58.353749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:08:58 2022] PUT /identity/v3/projects/16b9d419847e43258369829423c21fd0/users/b8d053796e4e443c849752ecb53fc9f2/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:09:00.448432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8d36d-0c13-4b3b-9608-a61e34144a8c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:00.449737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8d36d-0c13-4b3b-9608-a61e34144a8c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:00.449737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8d36d-0c13-4b3b-9608-a61e34144a8c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:00.450487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1154] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:09:00.456616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:00.457149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:00.457149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:00.476254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:00.477524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:00.552489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 05:09:00.577973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8d36d-0c13-4b3b-9608-a61e34144a8c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:00.760299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8d36d-0c13-4b3b-9608-a61e34144a8c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:00.823467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:00.824613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:00.825303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:00.825804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:00.830895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:00.849231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:09:00.860501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:09:00.861673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:00.862542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:09:00.863283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:09:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:09:00.870349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:00.907210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:00.992695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57967553-7159-47ab-a7a1-09d331cfc683 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:01.034817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:01.035395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:01.035676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:01.035957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:01.036682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:01.036994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:01.038412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:01.048950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:09:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:01.055522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:01.172358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c8100-ba3f-4f7f-94f5-b0ed2d4bc325 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:01.211543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:01.212399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:01.212788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:01.213169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] PATH_INFO: `/v3/users/b8d053796e4e443c849752ecb53fc9f2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:01.214725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] RBAC: Authorizing `identity:get_user(user_id=b8d053796e4e443c849752ecb53fc9f2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:01.216294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:01.218119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:09:01 2022] GET /identity/v3/users/b8d053796e4e443c849752ecb53fc9f2 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:09:01.224400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:01.324797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:01.332660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71f85b-3694-4fa6-9d6e-cad96bd0feb7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:01.377747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:01.378444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:01.378803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:01.379128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:01.451933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:01.453401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:01.555395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:01.565073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:09:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:09:01.575723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:01.686807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:01.755725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72acb-9114-44ce-a670-e55904345e71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:01.816066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb2ad2-c0b9-4f77-bb68-cac82413e0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:01.817190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb2ad2-c0b9-4f77-bb68-cac82413e0ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:01.817700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb2ad2-c0b9-4f77-bb68-cac82413e0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:01.818209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb2ad2-c0b9-4f77-bb68-cac82413e0ef None admin] PATH_INFO: `/v3/projects/16b9d419847e43258369829423c21fd0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:01.820326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb2ad2-c0b9-4f77-bb68-cac82413e0ef None admin] RBAC: Authorizing `identity:get_project(project_id=16b9d419847e43258369829423c21fd0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:01.823194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb2ad2-c0b9-4f77-bb68-cac82413e0ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:01.826765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:09:01 2022] GET /identity/v3/projects/16b9d419847e43258369829423c21fd0 => generated 311 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:01.832912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:01.949078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f4d89-cb67-4468-b78e-53a3412f61c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:01.988099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:01.988731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:01.989014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:01.989291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:01.990099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=16b9d419847e43258369829423c21fd0, user.id=b8d053796e4e443c849752ecb53fc9f2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:01.990410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:01.992047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d4bd0-85db-4c10-9d01-6bc3d9918c93 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:01.999527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:09:01 2022] GET /identity/v3/role_assignments?user.id=b8d053796e4e443c849752ecb53fc9f2&scope.project.id=16b9d419847e43258369829423c21fd0&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:04.145127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9397453-ef8f-4783-a730-faa9d870440e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:04.145676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9397453-ef8f-4783-a730-faa9d870440e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:04.145795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9397453-ef8f-4783-a730-faa9d870440e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:04.147598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1162] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:09:04.156459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:04.156967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:04.157412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:04.188900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:04.191038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:04.287365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:04.312405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:04.335884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9397453-ef8f-4783-a730-faa9d870440e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:04.450033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9397453-ef8f-4783-a730-faa9d870440e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:04.482682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:04.491163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:04.492284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:04.492803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:04.493291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:04.589782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:04.591248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:04.696178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:09:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:09:04.704259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:04.813408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1142f-734f-4b9c-ab6a-c69a8d504750 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:04.849251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:04.849907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:04.850193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:04.850473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:04.858692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:04.860714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-836bdb56-b4ef-4729-91f7-161422e16440 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:09:04.873366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:09:04.874645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:09:04.875521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:09:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:09:04.881495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:04.915831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:05.001008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968f76-f042-4665-9c04-b43377f70bb3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:05.042817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:05.043499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:05.043786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:05.044063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:05.044782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:05.045088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:05.046548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4762595-d24e-4e67-803a-bbb6c669dc55 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:05.055478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 05:09:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:07.171943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f223eb-3ec1-4795-8a85-7f013cb51e9a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:07.172749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f223eb-3ec1-4795-8a85-7f013cb51e9a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:07.172907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f223eb-3ec1-4795-8a85-7f013cb51e9a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:07.174594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1167] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:09:07.180421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46943ec-ea6d-4aa5-b538-943a1dc5afb1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:07.180949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46943ec-ea6d-4aa5-b538-943a1dc5afb1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:07.181059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46943ec-ea6d-4aa5-b538-943a1dc5afb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:07.201114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f46943ec-ea6d-4aa5-b538-943a1dc5afb1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:07.202350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f46943ec-ea6d-4aa5-b538-943a1dc5afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:07.291601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:07.314756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f223eb-3ec1-4795-8a85-7f013cb51e9a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:07.369549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:07.459137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f223eb-3ec1-4795-8a85-7f013cb51e9a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:07.502377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8f66a-b5c6-4dce-9022-e1d24d74169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:07.503027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8f66a-b5c6-4dce-9022-e1d24d74169e None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:07.503317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8f66a-b5c6-4dce-9022-e1d24d74169e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:07.503637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8f66a-b5c6-4dce-9022-e1d24d74169e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:07.504393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8f66a-b5c6-4dce-9022-e1d24d74169e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:07.505852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8f66a-b5c6-4dce-9022-e1d24d74169e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:07.523824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Fri Jul 29 05:09:07 2022] POST /identity/v3/domains => generated 267 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:09:09.733319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee866-2b77-40b5-9d45-7b6bf9b2b6d0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:09.733861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee866-2b77-40b5-9d45-7b6bf9b2b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:09.733954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee866-2b77-40b5-9d45-7b6bf9b2b6d0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:09.735455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1170] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:09:09.742089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:09.742617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:09.743133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:09.776007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:09.778128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:09.873521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:09.897091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ee866-2b77-40b5-9d45-7b6bf9b2b6d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:09.929254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:10.011999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ee866-2b77-40b5-9d45-7b6bf9b2b6d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:10.049344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58b51a-f031-4664-9741-9d2d45e2aaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:10.050000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58b51a-f031-4664-9741-9d2d45e2aaeb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:10.050283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58b51a-f031-4664-9741-9d2d45e2aaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:10.050560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58b51a-f031-4664-9741-9d2d45e2aaeb None admin] PATH_INFO: `/v3/domains/708e9939053e4db381cd1403d5823636` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:10.062624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58b51a-f031-4664-9741-9d2d45e2aaeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=708e9939053e4db381cd1403d5823636)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:10.064121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58b51a-f031-4664-9741-9d2d45e2aaeb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:10.065857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Jul 29 05:09:09 2022] GET /identity/v3/domains/708e9939053e4db381cd1403d5823636 => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:10.072033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:10.136393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:10.245962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d671f2-0f26-4dc9-9146-13930148f207 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:10.309262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9587-680c-4572-8366-1f3d6e07e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:10.310221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9587-680c-4572-8366-1f3d6e07e38c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:10.310725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9587-680c-4572-8366-1f3d6e07e38c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:10.311229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9587-680c-4572-8366-1f3d6e07e38c None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:10.312563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9587-680c-4572-8366-1f3d6e07e38c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:10.315377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9587-680c-4572-8366-1f3d6e07e38c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:10.363528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:09:10 2022] POST /identity/v3/projects => generated 361 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:09:12.505990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5657d-6bb8-4320-b28a-a0b673cf753b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:12.506507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5657d-6bb8-4320-b28a-a0b673cf753b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:12.506507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5657d-6bb8-4320-b28a-a0b673cf753b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:12.508059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1174] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:09:12.514462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:12.514954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:12.515120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:12.535304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:12.536632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:12.621314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:12.646817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5657d-6bb8-4320-b28a-a0b673cf753b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:12.739046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:12.757958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5657d-6bb8-4320-b28a-a0b673cf753b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:12.794440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c4436-81c7-4629-b70a-5d6aba96d9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:12.795184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c4436-81c7-4629-b70a-5d6aba96d9bd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:12.795560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c4436-81c7-4629-b70a-5d6aba96d9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:12.795889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c4436-81c7-4629-b70a-5d6aba96d9bd None admin] PATH_INFO: `/v3/domains/708e9939053e4db381cd1403d5823636` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:12.797132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c4436-81c7-4629-b70a-5d6aba96d9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=708e9939053e4db381cd1403d5823636)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:12.798891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c4436-81c7-4629-b70a-5d6aba96d9bd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:12.800595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Jul 29 05:09:12 2022] GET /identity/v3/domains/708e9939053e4db381cd1403d5823636 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:09:12.807061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:12.830453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:12.919303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a60ee-908d-43ff-959e-2f0ee1b8bd79 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:12.959713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84392d-49e6-41bf-b372-d412be656e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:12.960359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84392d-49e6-41bf-b372-d412be656e41 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:12.960649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84392d-49e6-41bf-b372-d412be656e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:12.960931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84392d-49e6-41bf-b372-d412be656e41 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:12.961636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84392d-49e6-41bf-b372-d412be656e41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:12.963757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84392d-49e6-41bf-b372-d412be656e41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:12.988288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:09:12 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:09:15.118363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352ae4c-3398-42e9-bbdd-8963d821032f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:15.119513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352ae4c-3398-42e9-bbdd-8963d821032f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:15.119513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352ae4c-3398-42e9-bbdd-8963d821032f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:15.121062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1178] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:09:15.126584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:15.127061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:15.127195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:15.147197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:15.148427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:15.221508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:15.245656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352ae4c-3398-42e9-bbdd-8963d821032f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:15.355193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352ae4c-3398-42e9-bbdd-8963d821032f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:15.391764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:15.392488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:15.392782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:15.393102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:15.405483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:09:15.410732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:09:15.411862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:09:15.412729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:09:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:09:15.422232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:15.432319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:15.546990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086cc15-565b-40df-8df3-b0c9bb978c27 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:15.588511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:15.589148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:15.589436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:15.589716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:15.590449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:15.590762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:15.592202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:15.601356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:09:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:15.610645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:15.628062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:15.723042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96e8c-8e44-4a48-b257-f5e732ce00d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:15.762341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:15.763056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:15.763420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:15.763760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] PATH_INFO: `/v3/users/0fa62e0ba9a845fa8742fe41f0c6f233` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:15.779443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] RBAC: Authorizing `identity:get_user(user_id=0fa62e0ba9a845fa8742fe41f0c6f233)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:15.781012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:15.782826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:09:15 2022] GET /identity/v3/users/0fa62e0ba9a845fa8742fe41f0c6f233 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:15.788943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:15.866539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:15.905522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f733cf4-d906-42ab-9829-a395caac4b7f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:15.941842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:15.942470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:15.942762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:15.943039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:16.018085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:16.019478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:16.097091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:16.136958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:09:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:09:16.143567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:16.233040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:16.268748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa068054-db40-43ff-9aab-1e45f6483325 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:16.308643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac0377-e8dc-4e1a-8952-83ecee790bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:16.309323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0377-e8dc-4e1a-8952-83ecee790bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:16.309613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0377-e8dc-4e1a-8952-83ecee790bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:16.309895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0377-e8dc-4e1a-8952-83ecee790bc2 None admin] PATH_INFO: `/v3/projects/9a7cce3c662d47acb13b94d23ec0c721` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:16.311148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac0377-e8dc-4e1a-8952-83ecee790bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=9a7cce3c662d47acb13b94d23ec0c721)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:16.312663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac0377-e8dc-4e1a-8952-83ecee790bc2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:16.314412 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:09:16 2022] GET /identity/v3/projects/9a7cce3c662d47acb13b94d23ec0c721 => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:16.320828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:16.429691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd318-1052-4271-8212-381beb1e5ce8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:16.468390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:16.469026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:16.469317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:16.469600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:16.470414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=9a7cce3c662d47acb13b94d23ec0c721, user.id=0fa62e0ba9a845fa8742fe41f0c6f233) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:16.470733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:16.472262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977205cb-425b-4938-b253-b5d111ffd80f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:16.478969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:09:16 2022] GET /identity/v3/role_assignments?user.id=0fa62e0ba9a845fa8742fe41f0c6f233&scope.project.id=9a7cce3c662d47acb13b94d23ec0c721&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:18.659145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0d06-20b3-4110-b857-be8a607fc1d4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:18.659848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0d06-20b3-4110-b857-be8a607fc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:18.659848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0d06-20b3-4110-b857-be8a607fc1d4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:18.661304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1186] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:09:18.667700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:18.668242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:18.668341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:18.689045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:18.690286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:18.718912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:18.764649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:18.788407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae0d06-20b3-4110-b857-be8a607fc1d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:18.923622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae0d06-20b3-4110-b857-be8a607fc1d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:18.960481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:18.961152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:18.961439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:18.961723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:18.971424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:18.975407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:09:18.981630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:09:18.983429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:09:18.985233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:09:18.990721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:19.102909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd444aa-c53c-41a5-b118-a89b9fc53af4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:19.141978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:19.142667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:19.143001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:19.143327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:19.144199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:19.144559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:19.146005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:19.156048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:19.162275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:19.277515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4593177-1877-4f5d-9863-525b494f7bcb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:19.316614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:19.317278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:19.317566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:19.317848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] PATH_INFO: `/v3/users/0fa62e0ba9a845fa8742fe41f0c6f233` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:19.319117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] RBAC: Authorizing `identity:get_user(user_id=0fa62e0ba9a845fa8742fe41f0c6f233)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:19.320629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:19.323281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:09:19 2022] GET /identity/v3/users/0fa62e0ba9a845fa8742fe41f0c6f233 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:09:19.332345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:19.361409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:19.443986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8ab46-db4c-44f5-980d-2b1180bfbf39 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:19.486243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a88a5-cd03-4fb7-8751-abebebb983e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:19.486822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a88a5-cd03-4fb7-8751-abebebb983e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:19.487113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a88a5-cd03-4fb7-8751-abebebb983e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:19.487422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a88a5-cd03-4fb7-8751-abebebb983e5 None admin] PATH_INFO: `/v3/projects/9a7cce3c662d47acb13b94d23ec0c721` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:19.488721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a88a5-cd03-4fb7-8751-abebebb983e5 None admin] RBAC: Authorizing `identity:get_project(project_id=9a7cce3c662d47acb13b94d23ec0c721)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:19.490234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a88a5-cd03-4fb7-8751-abebebb983e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:19.492105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:09:19 2022] GET /identity/v3/projects/9a7cce3c662d47acb13b94d23ec0c721 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:19.502321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:19.583678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:19.624633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e658d0d-2a7a-45fe-860e-6a9aa08202ae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:19.663248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a498cb2-c16e-42e2-b0ee-23177dd98382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:19.664021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498cb2-c16e-42e2-b0ee-23177dd98382 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:19.664248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498cb2-c16e-42e2-b0ee-23177dd98382 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:19.664531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498cb2-c16e-42e2-b0ee-23177dd98382 None admin] PATH_INFO: `/v3/projects/9a7cce3c662d47acb13b94d23ec0c721/users/0fa62e0ba9a845fa8742fe41f0c6f233/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:19.667119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a498cb2-c16e-42e2-b0ee-23177dd98382 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a7cce3c662d47acb13b94d23ec0c721, user_id=0fa62e0ba9a845fa8742fe41f0c6f233, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:19.668624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a498cb2-c16e-42e2-b0ee-23177dd98382 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:19.683970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:09:19 2022] PUT /identity/v3/projects/9a7cce3c662d47acb13b94d23ec0c721/users/0fa62e0ba9a845fa8742fe41f0c6f233/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:09:21.849424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddafe0-8a98-4d17-9d93-44e99185bf5d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:21.849995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddafe0-8a98-4d17-9d93-44e99185bf5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:21.849995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddafe0-8a98-4d17-9d93-44e99185bf5d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:21.851338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1193] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:09:21.858083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:21.858997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:21.858997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:21.878691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:09:21.879953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:09:21.960910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:09:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:09:21.985628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddafe0-8a98-4d17-9d93-44e99185bf5d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:22.094184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:22.104760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddafe0-8a98-4d17-9d93-44e99185bf5d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:22.146533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:22.147215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:22.147540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:22.147853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:22.171599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:09:22.177007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:09:22.178721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:09:22.180490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:09:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:09:22.189722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:22.262336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:22.313716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f9d13-a024-430b-bf9f-d5b3e97d369e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:22.353750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:22.354392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:22.354680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:22.355039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:22.355737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:22.356047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:22.357454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:22.366760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:22.374260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:22.485612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73b808-2a63-47a1-ba38-7bd59c764425 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:22.522884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:22.523579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:22.523865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:22.524144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] PATH_INFO: `/v3/users/0fa62e0ba9a845fa8742fe41f0c6f233` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:22.526563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] RBAC: Authorizing `identity:get_user(user_id=0fa62e0ba9a845fa8742fe41f0c6f233)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:22.528061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:22.529722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 05:09:22 2022] GET /identity/v3/users/0fa62e0ba9a845fa8742fe41f0c6f233 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:22.535879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:22.648143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b039ce-2432-4152-8622-fb78722b7fc7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:22.670688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:22.687616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:22.688256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:22.688541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:22.688821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:22.763438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:22.765036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:22.871194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:09:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 05:09:22.880520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:22.907527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:23.011108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61b58-1c1e-4bb0-9487-f986c683ebf3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:23.048592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a43044-2b25-4090-a177-b8eab308e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:23.049250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a43044-2b25-4090-a177-b8eab308e68d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:23.049536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a43044-2b25-4090-a177-b8eab308e68d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:23.049814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a43044-2b25-4090-a177-b8eab308e68d None admin] PATH_INFO: `/v3/projects/9a7cce3c662d47acb13b94d23ec0c721` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:23.050995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a43044-2b25-4090-a177-b8eab308e68d None admin] RBAC: Authorizing `identity:get_project(project_id=9a7cce3c662d47acb13b94d23ec0c721)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:23.052486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a43044-2b25-4090-a177-b8eab308e68d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:23.054193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 05:09:22 2022] GET /identity/v3/projects/9a7cce3c662d47acb13b94d23ec0c721 => generated 361 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:23.061378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:09:23.085034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:23.173411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51dd5-2b52-49cb-b578-88233bec5821 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:09:23.211601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:09:23.212876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:09:23.213368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:09:23.213825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:09:23.214625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=9a7cce3c662d47acb13b94d23ec0c721, user.id=0fa62e0ba9a845fa8742fe41f0c6f233) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:09:23.214945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:09:23.216595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426153b-338f-49fb-a4e2-1558a0dcab41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:09:23.223298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:09:23 2022] GET /identity/v3/role_assignments?user.id=0fa62e0ba9a845fa8742fe41f0c6f233&scope.project.id=9a7cce3c662d47acb13b94d23ec0c721&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:09:33.044107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=56 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:09:33.220312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:22.221235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569cedf-0aed-4df4-aa43-4e230bc09f1d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:22.221876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569cedf-0aed-4df4-aa43-4e230bc09f1d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:22.221876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569cedf-0aed-4df4-aa43-4e230bc09f1d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:22.223765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1201] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:22.229567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:22.230095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:22.230217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:22.252925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:22.254374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:22.340287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:22.364882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569cedf-0aed-4df4-aa43-4e230bc09f1d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:22.524410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569cedf-0aed-4df4-aa43-4e230bc09f1d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:22.581387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:22.582060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:22.582529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:22.582679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:22.595627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:22.597279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:22.617318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:22.618800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:22.620491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:22.620491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:22.620491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:22.620491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:22.620491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:10:22.625492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:22.755901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb952-e746-4087-b403-17f607ebe14d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:22.789901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:22.798292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:22.799267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:22.799946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:22.800478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:22.802004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:22.802325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:22.803992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783e72e-aae9-4e01-ad6f-8b4796d1d8f7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:22.824663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:22.824864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:22.911920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:22.974773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df0b8-b92f-477f-b028-11e33734afb1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:23.018540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20e19c-d154-4824-abc2-f6e79767d4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:23.019170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20e19c-d154-4824-abc2-f6e79767d4c2 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:23.019597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20e19c-d154-4824-abc2-f6e79767d4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:23.019904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20e19c-d154-4824-abc2-f6e79767d4c2 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:23.021032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20e19c-d154-4824-abc2-f6e79767d4c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:23.022108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20e19c-d154-4824-abc2-f6e79767d4c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:23.055939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 05:10:22 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:10:25.437457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26e35c-c437-4b1f-ae08-06ee05050e7f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:25.438016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26e35c-c437-4b1f-ae08-06ee05050e7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:25.438118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26e35c-c437-4b1f-ae08-06ee05050e7f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:25.439503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1206] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:25.445843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:25.446289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:25.448532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:25.468910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:25.470116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:25.548849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:25.575230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26e35c-c437-4b1f-ae08-06ee05050e7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:25.706455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26e35c-c437-4b1f-ae08-06ee05050e7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:25.750853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:25.751532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:25.755571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:25.755900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:25.768996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:10:25.775591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:25.776691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:25.777570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:10:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:25.783441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:25.831493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:25.904743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb04cc-d7ad-4e9e-8d02-635f4de0812c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:25.944300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:25.945161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:25.945325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:25.945519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:25.946252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:25.946551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:25.947987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:25.964567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:10:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:25.966015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:26.033716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:26.097009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ccdad-1b41-4227-8311-fabb986f298c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:26.139020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:26.139786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:26.140131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:26.140454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:26.152784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:26.154600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:26.171626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:26.172818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:26.173911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:26.173911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:26.173911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:26.173911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:26.175514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:26.179703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:26.313888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:26.318029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2bcd-04e8-477f-871c-31748fbc0939 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:26.359989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:26.360804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:26.361130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:26.361450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:26.362157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:26.362479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:26.364670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:26.382229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:10:26.383563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:26.505525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892eae7-8541-42a2-b21a-c7ac662e6de3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:26.508157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:26.548265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:26.549121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:26.549510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:26.549894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:26.559149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:26.561016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:10:26.574631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:26.579503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:26.584057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:26.585899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:26.712812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1b0c4-5d60-4fd8-a057-60595ba97018 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:26.751573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:26.752314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:26.752602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:26.753009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:26.754489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:26.755480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:26.756945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:26.777079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 05:10:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:26.782870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:26.904729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf5d1-b3ff-4af6-b338-0c9b5291bc97 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:26.945471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:26.946141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:26.946430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:26.946709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:26.986830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:27.038671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:27.040231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:27.172120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:10:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 05:10:27.173981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:27.204923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:27.329714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c79e4-881c-4feb-aa2e-b70b07dc61c4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:27.370504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:27.371194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:27.371507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:27.371791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:27.386227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:27.387965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:27.407999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:27.409405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:27.410770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:27.410770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:27.410770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:27.410770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:27.411606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:27.417385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:27.496222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:27.571752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fefd63-42f6-4dab-be6d-9593067fd110 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:27.616277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:27.617036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:27.617470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:27.617781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:27.618709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:27.619024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:27.620629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:27.641277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:10:27.642810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:27.724679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:27.778362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fb0b8-c8fb-4850-892f-bbca6a1be63d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:27.818475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186474-58b1-455f-8b01-cce521555cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:27.819233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186474-58b1-455f-8b01-cce521555cda None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:27.819613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186474-58b1-455f-8b01-cce521555cda None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:27.819949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186474-58b1-455f-8b01-cce521555cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:27.827722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01186474-58b1-455f-8b01-cce521555cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:27.829290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01186474-58b1-455f-8b01-cce521555cda None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-01186474-58b1-455f-8b01-cce521555cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:10:27.841660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:27.842848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:27.849459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:10:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:10:27.850096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:27.980816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce391943-ea4e-4c7a-ba48-e7e759b15a55 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:28.009191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:28.028177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f14699-4a17-47df-afee-786757c7662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:28.028903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f14699-4a17-47df-afee-786757c7662d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:28.029201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f14699-4a17-47df-afee-786757c7662d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:28.029606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f14699-4a17-47df-afee-786757c7662d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:28.030463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f14699-4a17-47df-afee-786757c7662d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:28.030981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f14699-4a17-47df-afee-786757c7662d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:28.032435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f14699-4a17-47df-afee-786757c7662d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:28.051410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:10:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:10:28.052931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186474-58b1-455f-8b01-cce521555cda None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:28.185489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186474-58b1-455f-8b01-cce521555cda None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:28.227975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:28.235870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:28.236656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:28.236998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:28.237576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:28.239157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=5df6c21c2ed44bf0b07cd16454b3e0ed) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:28.239531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:28.241565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460133e-23d8-40fd-82e3-a6e63d151e91 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:28.248070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:10:28 2022] GET /identity/v3/role_assignments?user.id=5df6c21c2ed44bf0b07cd16454b3e0ed&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:30.575921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036719-8c86-403c-aa58-e8ae6644a1a5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:30.576440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036719-8c86-403c-aa58-e8ae6644a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:30.576440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036719-8c86-403c-aa58-e8ae6644a1a5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:30.577818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1220] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:30.584101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:30.584570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:30.584671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:30.605041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:30.606290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:30.693118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:30.717227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c036719-8c86-403c-aa58-e8ae6644a1a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:30.838225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c036719-8c86-403c-aa58-e8ae6644a1a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:30.882189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:30.882947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:30.883268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:30.883670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:30.899402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:10:30.904480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:30.905711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:30.906790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:10:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:30.912595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:31.024163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7987f30-a3ac-4202-8011-49c7a6077e32 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:31.053860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:31.067246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:31.067910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:31.068193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:31.068472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:31.069195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:31.069544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:31.071468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:31.083374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:10:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:31.089273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:31.144093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:31.235103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42a8c-80c5-436f-ae52-9be57640aec3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:31.290477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:31.291143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:31.291477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:31.291769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:31.303846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:31.305622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:31.322453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:31.323580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:31.329807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:31.329807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:31.329807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:31.329807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:31.329807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:31.330239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:31.445739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799a761-dd40-4edc-9174-eacd9ecbd7dd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:31.485183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:31.485940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:31.486287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:31.486610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:31.487741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:31.488062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:31.489775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:31.504848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:31.511153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:31.629843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:31.638957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43eb38-e2aa-481f-8a69-33f959bd6c0d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:31.678424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:31.679097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:31.679410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:31.679697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:31.689537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:31.691941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:10:31.705960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:31.708005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:31.709355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:31.713975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:31.769361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:31.830250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a98cfd-25fb-4d0f-8c73-c9f985537c8b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:31.867991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:31.868802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:31.869121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:31.869454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:31.870469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:31.870862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:31.872754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:31.896525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 05:10:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:31.902851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:32.017062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae32d8-5cbe-4633-a97e-ab42c124b8f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:32.056003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:32.056933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:32.057300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:32.057585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:32.071782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:32.073559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:32.090775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:32.091948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:32.093123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:32.093123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:32.093123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:32.093123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:32.093123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:32.098773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:32.213062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa6e2d-2676-40f8-b236-ad26b2c0fc05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:32.215644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:32.251032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:32.251687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:32.251967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:32.252238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:32.252922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:32.253233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:32.255227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:32.266940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:32.273842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:32.288810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:32.396935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176991f-f60b-4d99-9ce3-91226d80dbba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:32.433648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:32.434411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:32.434727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:32.435083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:32.442927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:32.444534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:10:32.456449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:32.458154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:32.459231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:32.463627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:10:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:32.464639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:32.578006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9d157-2f76-40d2-92f0-47bc3a903c13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:32.614820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:32.615574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:32.615860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:32.616152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:32.617097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:32.617448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:32.618930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18fca8-b019-44bd-94c1-99e428c6dd69 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:32.633086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:10:32.639690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:32.721991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:32.766498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cb496-19bc-4c45-a7ce-942f1aeb63e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:32.807920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77c82d-c946-475a-937a-28e2ff2e0bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:32.808624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77c82d-c946-475a-937a-28e2ff2e0bf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:32.808915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77c82d-c946-475a-937a-28e2ff2e0bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:32.809202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77c82d-c946-475a-937a-28e2ff2e0bf0 None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/5df6c21c2ed44bf0b07cd16454b3e0ed/roles/77f4411eb7384acc86f80c0fb72ff3f7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:32.825725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77c82d-c946-475a-937a-28e2ff2e0bf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=5df6c21c2ed44bf0b07cd16454b3e0ed, role_id=77f4411eb7384acc86f80c0fb72ff3f7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:32.827444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77c82d-c946-475a-937a-28e2ff2e0bf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:32.842766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:10:32 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/5df6c21c2ed44bf0b07cd16454b3e0ed/roles/77f4411eb7384acc86f80c0fb72ff3f7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:10:34.925734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4912-dcaa-4dea-b134-63e9ca5588c4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:34.926296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4912-dcaa-4dea-b134-63e9ca5588c4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:34.926398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4912-dcaa-4dea-b134-63e9ca5588c4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:34.927733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1233] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:34.933170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:34.933641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:34.933749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:34.952844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:34.954057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:35.026225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:35.049769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4912-dcaa-4dea-b134-63e9ca5588c4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:35.076861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:35.161342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4912-dcaa-4dea-b134-63e9ca5588c4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:35.201106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:35.201765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:35.202054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:35.202346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:35.214255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:10:35.219144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:35.220275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 05:10:35.221168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:35.223514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 05:10:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:35.226616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:35.277934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:35.343499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf06f9-34f4-4bb4-ac59-29caf2f65dd1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:35.382894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:35.383559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:35.383840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:35.384116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:35.384834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:35.385142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:35.386522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:35.395052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 05:10:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:35.400862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:35.491104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:35.509219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccafca-80a8-4e63-bdc5-88a0d40d73af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:35.547828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:35.548484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:35.548774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:35.549056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:35.560139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:35.561611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:35.578082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:35.579110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:35.580418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:35.580418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:35.580418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:35.580418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:35.580418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:35.585219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:35.700516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e09c-e1a6-4421-bed8-9a9e1d5a66d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:35.710563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:35.739852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:35.740488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:35.740778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:35.741056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:35.741804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:35.742057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:35.743469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:35.756564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:35.762258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:35.888385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a65af-abc9-461f-94d8-0bc2f7b77f2c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:35.939458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:35.940360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:35.941266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:35.941669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:35.945389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:35.953171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:35.954836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:10:35.967749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:35.968972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:35.970290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:35.975194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:36.085342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b19261-f76d-4f3f-ba9e-9c1469897692 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:36.122954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:36.123698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:36.124024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:36.124353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:36.125182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:36.125532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:36.127024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:36.146663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 05:10:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:36.152687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:36.282782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b3c2-5445-4e60-9601-045931d3d6e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:36.328247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:36.329298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:36.329812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:36.330250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:36.430995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:36.439314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:36.441353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:36.567181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:10:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 05:10:36.573187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:36.603756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:36.687962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf5717-a620-4497-812f-d462fd779f26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:36.741580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:36.742240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:36.742553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:36.742803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:36.756559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:36.757970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:36.773995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:36.775218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:36.776444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:36.776444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:36.776444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:36.776444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:36.776444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:36.780955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:36.902940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:36.910143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8355-4271-49b5-a15b-7e437af2caa3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:36.955090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:36.955804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:36.956138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:36.956570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:36.957463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:36.957795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:36.959265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:36.973123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:36.979712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:37.111411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:37.136427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced50a1e-2ed7-4ff4-84b5-e4114de4d3ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:37.182761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:37.183464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:37.183753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:37.184070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:37.192058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:37.193622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:10:37.207462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:37.209590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:37.212109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:37.218350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:37.340098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7fc31-5498-4819-a280-d0af010bbc03 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:37.380452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:37.381206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:37.381544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:37.381832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:37.382586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:37.382921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:37.384550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5634c1-a0cd-4480-ab1d-93f1d00c25c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:37.401384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:10:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:37.408346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:37.523208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf6b0d-1b03-4a8f-831e-fb17828b870f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:37.562414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:37.563101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:37.563430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:37.563719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:37.564672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] RBAC: Adding query filter params (role.id=77f4411eb7384acc86f80c0fb72ff3f7, scope.project.id=cd00b15fba514550924237d11784db71, user.id=5df6c21c2ed44bf0b07cd16454b3e0ed) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:37.565057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:37.567464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a787b0e-1b8a-4e1a-8b96-f08eaabccc1c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:37.574407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:10:37 2022] GET /identity/v3/role_assignments?user.id=5df6c21c2ed44bf0b07cd16454b3e0ed&scope.project.id=cd00b15fba514550924237d11784db71&role.id=77f4411eb7384acc86f80c0fb72ff3f7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:40.072014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed8737-59fb-4cb0-ada0-4da3038e312d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:40.072014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed8737-59fb-4cb0-ada0-4da3038e312d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:40.072014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed8737-59fb-4cb0-ada0-4da3038e312d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:40.072014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1247] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:40.078460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:40.078971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:40.079092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:40.087143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:40.101079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:40.102436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:40.179693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:40.204386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed8737-59fb-4cb0-ada0-4da3038e312d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:40.224894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:40.321305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed8737-59fb-4cb0-ada0-4da3038e312d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:40.359139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:40.359776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:40.360098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:40.360341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:40.373465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:10:40.378923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:40.380873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:40.382499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:10:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:40.386945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:40.508170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08351c1-5270-4a3a-b54d-29135de5282a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:40.545932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:40.551983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515344-5479-42d8-b624-63517122944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:40.552667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515344-5479-42d8-b624-63517122944b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:40.553030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515344-5479-42d8-b624-63517122944b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:40.553277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515344-5479-42d8-b624-63517122944b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:40.554094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515344-5479-42d8-b624-63517122944b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:40.554423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515344-5479-42d8-b624-63517122944b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:40.555964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515344-5479-42d8-b624-63517122944b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:40.565788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:10:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:10:40.572250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:40.665383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:40.695055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bf4bb-c58c-436a-95e1-fb1ab0e74273 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:40.740885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:40.743037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:40.743037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:40.743037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:40.755456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:40.756933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:40.774930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:40.776157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:40.777108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:40.777108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:40.777108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:40.777108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:40.777108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:40.782296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515344-5479-42d8-b624-63517122944b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:40.898716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515344-5479-42d8-b624-63517122944b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:40.940947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:40.941883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:40.942187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:40.942448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:40.943170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:40.943519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:40.944958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:40.959162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:40.965032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:41.086825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56cef0-a24d-46af-95d2-6c0396d2c3bc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:41.116651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:41.126365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:41.127073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:41.127414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:41.128278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:41.136999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:41.138461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:10:41.154572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:41.155890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:41.157017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:41.161964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:41.230814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:41.286685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5a517-fb7a-413c-a5f1-1d4cb98e8cf2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:41.330681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:41.331535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:41.331831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:41.332117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:41.332921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:41.333236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:41.334688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:41.353516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 05:10:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:41.359482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:41.485001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816fa67-8e92-44c8-a221-b874d96585e8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:41.526311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:41.526950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:41.527239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:41.527555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:41.596496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:41.609871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:41.611333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:41.725903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:10:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:10:41.731709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:41.841254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1a0fd-63ea-44d8-979b-d5a0f22fa3cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:41.857938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:41.888768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:41.889600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:41.890049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:41.890761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:41.905197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:41.906817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:41.926550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:41.927815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:41.928811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:41.928811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:41.928811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:41.928811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:41.928811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:41.934672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:42.030317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:42.048833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721dedb-4044-4fcf-a5bf-c254ae4c1bb4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:42.093592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:42.094303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:42.094398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:42.094674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:42.095537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:42.095854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:42.098019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:42.111279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:42.118284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:42.253125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064eb75-fe0e-4a6d-93d9-ebd12e262331 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:42.295042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:42.296148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:42.296490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:42.296891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:42.308960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:42.310682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:10:42.325657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:42.326654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:42.327574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 05:10:42.334304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:42.459266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e449-9dc4-40e4-9676-b9b17631be37 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:42.500325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:42.501101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:42.501429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:42.501727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:42.502654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:42.503334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:42.504878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2c3f-4775-4b9e-8ed4-fcabe7d27501 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:42.519840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:42.527049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:42.558256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:42.644946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471197a3-0efa-46d9-a7ba-698c9b70b7b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:42.684769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:42.685483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:42.685773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:42.686052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:42.686925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=cd00b15fba514550924237d11784db71, user.id=5df6c21c2ed44bf0b07cd16454b3e0ed) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:42.687236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:42.688716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f1879-1008-4151-83a9-8bead9247499 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:42.695488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:10:42 2022] GET /identity/v3/role_assignments?user.id=5df6c21c2ed44bf0b07cd16454b3e0ed&scope.project.id=cd00b15fba514550924237d11784db71&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:44.771950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910144d0-0fce-453c-9d90-9e61a68cb473 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:44.772516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910144d0-0fce-453c-9d90-9e61a68cb473 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:44.772516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910144d0-0fce-453c-9d90-9e61a68cb473 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:44.773975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1261] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:44.779105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:44.779685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:44.779830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:44.798237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:44.799966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:44.865784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:44.891917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910144d0-0fce-453c-9d90-9e61a68cb473 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:44.905365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:45.008838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910144d0-0fce-453c-9d90-9e61a68cb473 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:45.045776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:45.046415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:45.046699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:45.046981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:45.058615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:10:45.063592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:45.065432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:45.066967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:45.073347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:45.090646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:45.178033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1616538-e8e7-4fee-a026-972efe9a813a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:45.211783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:45.212411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:45.212690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:45.212963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:45.213671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:45.213979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:45.215381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:45.223999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:45.229813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:45.295877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:45.329356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a30b9a-1bd5-44ed-a9c0-005d7d058ba2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:45.364972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:45.365626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:45.365914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:45.366194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:45.376575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:45.378549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:45.393976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:45.395216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:45.396317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:45.396317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:45.396317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:45.396317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:45.396317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:45.401113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:45.485216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:45.506453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416722dd-dc97-4832-a53d-4b629020e91b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:45.540361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:45.540986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:45.541272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:45.541553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:45.542245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:45.542553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:45.544224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:45.556919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:45.562524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:45.667075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe59af-ee46-487f-8d94-a4786eb87828 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:45.687746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:45.702459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:45.703106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:45.703417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:45.703699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:45.724009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:45.725493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:10:45.738726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:45.739950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:45.740912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:45.746098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:45.851910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fa137-fdcb-4459-ad0d-e1ffa8b0a72f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:45.890871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:45.891567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:45.891926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:45.892223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:45.893095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:45.893418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:45.895166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:45.906014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:45.917799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 05:10:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:10:45.923806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:46.042828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c959b-bf79-4652-bf9e-d4520a8d52e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:46.077560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:46.078227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:46.078494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:46.078772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:46.089792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:46.091249 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:46.106045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:46.107120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:46.108337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:46.108337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:46.108337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:46.108337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:46.108337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:46.113508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:46.220186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683d955-c114-433e-807d-25a9df1c8673 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:46.256819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:46.257446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:46.257730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:46.258011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:46.258703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:46.259013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:46.260937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:46.273701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:46.279602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:46.314087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:46.388784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a22f3-5c37-48f8-9ede-1758c41acfff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:46.423889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:46.425059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:46.425344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:46.425617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:46.432550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:46.434007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:10:46.446093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:46.447298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:46.448261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:10:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:46.453136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:46.511962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:46.555030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cff398-0316-4387-b0a5-f682003e9042 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:46.589160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:46.589780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:46.590063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:46.590337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:46.591074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:46.591410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:46.592846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f85116-5db7-43a7-8d81-7184f993b1ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:46.605102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:10:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:46.611058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:46.701838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:46.710818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ac43-a1ac-4bb8-b82d-de9592e78d8c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:46.745020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4400a2b-1fb3-4e93-90b2-83fd1f08811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:46.745684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4400a2b-1fb3-4e93-90b2-83fd1f08811e None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:46.745965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4400a2b-1fb3-4e93-90b2-83fd1f08811e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:46.746239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4400a2b-1fb3-4e93-90b2-83fd1f08811e None admin] PATH_INFO: `/v3/projects/cd00b15fba514550924237d11784db71/users/5df6c21c2ed44bf0b07cd16454b3e0ed/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:46.748149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4400a2b-1fb3-4e93-90b2-83fd1f08811e None admin] RBAC: Authorizing `identity:create_grant(project_id=cd00b15fba514550924237d11784db71, user_id=5df6c21c2ed44bf0b07cd16454b3e0ed, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:46.749584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4400a2b-1fb3-4e93-90b2-83fd1f08811e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:46.764028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 05:10:46 2022] PUT /identity/v3/projects/cd00b15fba514550924237d11784db71/users/5df6c21c2ed44bf0b07cd16454b3e0ed/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:10:48.756575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63205b-db61-4ac6-be34-c25fa1bee352 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:48.757117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63205b-db61-4ac6-be34-c25fa1bee352 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:48.757211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63205b-db61-4ac6-be34-c25fa1bee352 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:48.758512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1274] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:48.763738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:48.764170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:48.764308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:48.782806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:48.784645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:48.857009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:48.885635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63205b-db61-4ac6-be34-c25fa1bee352 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:49.004155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63205b-db61-4ac6-be34-c25fa1bee352 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:49.021391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:49.039833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:49.040490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:49.040778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:49.041064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:49.052376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 05:10:49.057330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:49.058454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:49.064343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 05:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:49.065513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:49.147084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:49.166971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443ec20-92ed-4329-bbe8-b0136712eaaa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:49.203376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:49.204028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:49.204314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:49.204594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:49.205838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:49.206148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:49.207564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:49.216395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 05:10:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:49.222529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:49.323128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc311912-7d59-494e-a8fe-4b77d2641ac1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:49.359586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:49.360264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:49.360547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:49.360828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:49.374215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:49.375729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:49.392112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:49.393949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:49.395266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:49.395266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:49.395266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:49.395266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:49.395266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:49.400283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:49.503583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd639be-de44-4b24-b4cb-7d7d1a2f55d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:49.541701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:49.542343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:49.542634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:49.542914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:49.543908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:49.544219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:49.545622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:49.559004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:49.565362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:49.599176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:49.677150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc17ff1-576f-465f-99e5-f715f90b08b7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:49.715671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:49.716328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:49.716912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:49.716912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:49.726204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:49.727738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 05:10:49.740091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:49.741302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:49.742299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:49.748298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:49.753528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:49.853470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089b9bd-7f0b-488e-bf1d-bd7989dc6bfe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:49.894448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:49.895191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:49.895569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:49.895970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:49.896829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:49.897138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:49.898678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:49.916323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 05:10:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:49.922371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:50.017229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:50.029222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d3640-575d-4384-8ba7-1ac5f128d847 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:50.068752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:50.069402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:50.069691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:50.069973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:50.139104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:50.140620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:50.232062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:50.242544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 05:10:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 05:10:50.248383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:50.310546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:50.353536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85024140-cbe4-4ca4-8fae-bb9053fd18d9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:50.388589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:50.389288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:50.389519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:50.389795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:50.400034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:50.401495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 05:10:50.417136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:50.418201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:50.419413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 05:10:50.419413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 05:10:50.419413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 05:10:50.419413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:50.419413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:50.425121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:50.524913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cadde0-0987-4dfd-89dc-b6891087eb21 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:50.560000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:50.560623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:50.560908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:50.561188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:50.561876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:50.562207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:50.563630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:50.575611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 05:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:50.581880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:50.685466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedcbd1-18e4-47a7-9e1d-d4146a7f3420 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:50.699139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:50.719926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:50.720614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:50.721475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:50.721745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:50.728948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:50.730404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 05:10:50.741914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:50.743015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 29 05:10:50.744363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 05:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:50.748993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:50.800940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:50.853235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad25a5-e391-469b-813b-af0e4ff95d48 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:50.890311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:50.891061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:50.891285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:50.891596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:50.892952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:50.893449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:50.895088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664e398-2a63-4532-9366-fb887a57a24b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:50.907458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 05:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:50.914699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:51.013969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55051a-46c4-4828-8b6c-e8de15046713 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:51.048142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:51.048984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:51.049150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:51.049383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:51.050228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] RBAC: Adding query filter params (role.id=8b45fb0997304e788e2bb021f59cc269, scope.project.id=cd00b15fba514550924237d11784db71, user.id=5df6c21c2ed44bf0b07cd16454b3e0ed) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:51.050536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:51.052002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146eaa3-ea2e-407b-92c3-b0b32154abd8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:51.057812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 05:10:50 2022] GET /identity/v3/role_assignments?user.id=5df6c21c2ed44bf0b07cd16454b3e0ed&scope.project.id=cd00b15fba514550924237d11784db71&role.id=8b45fb0997304e788e2bb021f59cc269 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:53.134372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8bd09-8465-45c2-b287-8ec002a3e8ef None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:53.135053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8bd09-8465-45c2-b287-8ec002a3e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:53.135053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8bd09-8465-45c2-b287-8ec002a3e8ef None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:53.136708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1288] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:53.142634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:53.143133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:53.143223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:53.162945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:53.164304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:53.200117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:53.242935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:53.266284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8bd09-8465-45c2-b287-8ec002a3e8ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:53.391767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8bd09-8465-45c2-b287-8ec002a3e8ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:53.407081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:53.435209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:53.435935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:53.436219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:53.436501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:53.437149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:53.438589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:53.449857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:53.451730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:53.453036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:10:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:53.458526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:53.585907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d462e-5e02-45bc-b3f7-56b02b774e75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:53.627775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:53.628445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:53.628744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:53.629053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:53.629784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:53.630126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:53.631606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b76d5-8462-4b52-93dc-9246aac8bf88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:53.637316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 05:10:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:53.644584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:53.774983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f73c2-8efc-4c2c-9808-151a020972a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:53.828043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:53.828743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:53.829059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:53.829410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:53.830246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:53.830619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:53.832052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3734eb-ad7b-4b2a-b37c-f9db957ccde6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:53.837551 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 05:10:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:56.014939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69901-bc88-4da9-8eb3-481c3fde33af None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:56.015561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69901-bc88-4da9-8eb3-481c3fde33af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:56.015561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69901-bc88-4da9-8eb3-481c3fde33af None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:56.016917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1293] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:56.022305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63241f-917b-44a4-8d79-03fae1330202 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:56.022756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63241f-917b-44a4-8d79-03fae1330202 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:56.022882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63241f-917b-44a4-8d79-03fae1330202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:56.041233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cb63241f-917b-44a4-8d79-03fae1330202 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:56.042436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb63241f-917b-44a4-8d79-03fae1330202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:56.074696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:56.113995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:56.136639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad69901-bc88-4da9-8eb3-481c3fde33af None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:56.167438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:56.240499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad69901-bc88-4da9-8eb3-481c3fde33af None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:56.275037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de7902-bf72-443a-9ea9-103226a714d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:56.275609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de7902-bf72-443a-9ea9-103226a714d9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:56.275901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de7902-bf72-443a-9ea9-103226a714d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:56.276186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de7902-bf72-443a-9ea9-103226a714d9 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:56.276881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de7902-bf72-443a-9ea9-103226a714d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:56.278286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de7902-bf72-443a-9ea9-103226a714d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:56.298787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 05:10:56 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:10:58.451227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a29a-e879-4276-a9ea-9cc39653099c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:58.451783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a29a-e879-4276-a9ea-9cc39653099c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:58.451783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a29a-e879-4276-a9ea-9cc39653099c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:58.453119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1296] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:10:58.459683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:58.460172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:58.460300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:58.480272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:10:58.481486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:10:58.565363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:10:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:10:58.589388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48a29a-e879-4276-a9ea-9cc39653099c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:58.706611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48a29a-e879-4276-a9ea-9cc39653099c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:58.744320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:58.745096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:58.745433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:58.745763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:58.746537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:58.748092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:10:58.758266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:10:58.759581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:10:58.760672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:10:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:10:58.766038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:58.788164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:58.888591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274bf5-483e-487a-84d7-c759dd1f9871 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:58.935151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269386-f424-4396-aca9-0a669f61024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:58.935824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269386-f424-4396-aca9-0a669f61024f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:58.936115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269386-f424-4396-aca9-0a669f61024f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:58.936396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269386-f424-4396-aca9-0a669f61024f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:58.937141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269386-f424-4396-aca9-0a669f61024f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:58.937486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269386-f424-4396-aca9-0a669f61024f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:58.938882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269386-f424-4396-aca9-0a669f61024f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:58.946139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 05:10:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:58.952440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:58.986695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:59.071704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a864816-7d20-4c1d-9c46-516bfe42e946 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:59.123724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:59.124348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:59.124632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:59.124909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:59.125670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] RBAC: Adding query filter params (interface=public, service_id=8088c0f12869490696ecf34218096ac3, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:10:59.125978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:59.127651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4e149-fc50-40b1-bf97-a2d0c429e62d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:59.133191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 05:10:58 2022] GET /identity/v3/endpoints?service_id=8088c0f12869490696ecf34218096ac3&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:10:59.139972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269386-f424-4396-aca9-0a669f61024f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:10:59.233430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:10:59.254780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269386-f424-4396-aca9-0a669f61024f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:10:59.305463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddfe2-9f9b-40fd-99c1-4decf7a15820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:10:59.306120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddfe2-9f9b-40fd-99c1-4decf7a15820 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:10:59.306431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddfe2-9f9b-40fd-99c1-4decf7a15820 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:10:59.306715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddfe2-9f9b-40fd-99c1-4decf7a15820 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:10:59.307397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baddfe2-9f9b-40fd-99c1-4decf7a15820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:10:59.308793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baddfe2-9f9b-40fd-99c1-4decf7a15820 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:10:59.316806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 05:10:59 2022] GET /identity/v3/services => generated 2191 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:11:01.464395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a7c78-b8f5-4ff5-8344-e1bd0dd46b66 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:11:01.464913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a7c78-b8f5-4ff5-8344-e1bd0dd46b66 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:11:01.464913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a7c78-b8f5-4ff5-8344-e1bd0dd46b66 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:11:01.466238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1302] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:11:01.471276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:11:01.471714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:11:01.471881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:11:01.489790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:11:01.490986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:11:01.557689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:11:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 05:11:01.581079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a7c78-b8f5-4ff5-8344-e1bd0dd46b66 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:11:01.677006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:11:01.684182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a7c78-b8f5-4ff5-8344-e1bd0dd46b66 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:11:01.717707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:11:01.718274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:11:01.718559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:11:01.718839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:11:01.719498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:11:01.721684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 05:11:01.730669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 05:11:01.731916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 29 05:11:01.733204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 05:11:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 05:11:01.738373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:11:01.814114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:11:01.837694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868b55a-a5c7-4537-b8f1-4cb0ade3340a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:11:01.871296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:11:01.871950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:11:01.872238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:11:01.872518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:11:01.873199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:11:01.873509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:11:01.874895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055fe26-e7c4-44ca-95e6-a6fd742c5406 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:11:01.880035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 05:11:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:11:01.885847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:11:01.984901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6c328-f812-45c7-859a-ebc4a947e274 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:11:02.018078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394e61-295a-4fa5-985f-d43fbd66080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:11:02.018698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394e61-295a-4fa5-985f-d43fbd66080c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:11:02.018986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394e61-295a-4fa5-985f-d43fbd66080c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:11:02.019265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394e61-295a-4fa5-985f-d43fbd66080c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:11:02.020009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394e61-295a-4fa5-985f-d43fbd66080c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:11:02.021388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394e61-295a-4fa5-985f-d43fbd66080c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:11:02.042220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:11:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:11:11.877061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=30 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:11:12.037697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:09.266165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013edf6c-5ddf-447e-8f04-45c41a4132f6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:09.267225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013edf6c-5ddf-447e-8f04-45c41a4132f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:09.267774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013edf6c-5ddf-447e-8f04-45c41a4132f6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:09.269583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1307] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:09.275173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a81c-dd4a-4bb6-b492-e309ec0f86cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:09.276224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a81c-dd4a-4bb6-b492-e309ec0f86cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:09.277273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a81c-dd4a-4bb6-b492-e309ec0f86cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:09.445281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9f41a81c-dd4a-4bb6-b492-e309ec0f86cc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:09.446966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f41a81c-dd4a-4bb6-b492-e309ec0f86cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:09.656438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:09.751738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1928b-4fdc-42a9-90ac-3ad2493d2c74 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:09.752251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1928b-4fdc-42a9-90ac-3ad2493d2c74 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:09.752774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1928b-4fdc-42a9-90ac-3ad2493d2c74 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:09.754719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1309] 10.222.0.49 () {54 vars in 965 bytes} [Fri Jul 29 05:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:09.764140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c4c8d-e39f-43ad-9f51-2300d9c9a214 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:09.764758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c4c8d-e39f-43ad-9f51-2300d9c9a214 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:09.765259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c4c8d-e39f-43ad-9f51-2300d9c9a214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:09.945561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-581c4c8d-e39f-43ad-9f51-2300d9c9a214 None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:09.948549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-581c4c8d-e39f-43ad-9f51-2300d9c9a214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:10.000137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:10.080648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Jul 29 05:12:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:12:10.090646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1928b-4fdc-42a9-90ac-3ad2493d2c74 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:10.092646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bd1928b-4fdc-42a9-90ac-3ad2493d2c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:10.284000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1928b-4fdc-42a9-90ac-3ad2493d2c74 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:10.398532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:10.399188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:10.399513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:10.399790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:10.401552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:10.467787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:10.562071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:10.564846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae600be-567e-4b54-b76c-28de3ae4e847 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:10.593879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:13.514276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b857ee-31dd-4c64-a3c1-df699051d715 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:13.514788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b857ee-31dd-4c64-a3c1-df699051d715 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:13.514788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b857ee-31dd-4c64-a3c1-df699051d715 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:13.516450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1312] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:13.522217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2c070-76cf-46f7-a110-827311c603fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:13.522727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2c070-76cf-46f7-a110-827311c603fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:13.522832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2c070-76cf-46f7-a110-827311c603fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:13.617731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7c2c070-76cf-46f7-a110-827311c603fe None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:13.618961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7c2c070-76cf-46f7-a110-827311c603fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:13.718458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:13.756913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b857ee-31dd-4c64-a3c1-df699051d715 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:13.758474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16b857ee-31dd-4c64-a3c1-df699051d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:13.866278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b857ee-31dd-4c64-a3c1-df699051d715 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:13.939872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:13.942442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:13.943085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:13.943600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:13.944073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:13.945922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:14.060826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:14.062275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340c0c6-0788-4cf3-9e50-d651497c6020 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:14.081750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:16.451415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a4aec-1bbe-42b1-a71d-7f9f020e12fb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:16.451927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a4aec-1bbe-42b1-a71d-7f9f020e12fb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:16.451927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a4aec-1bbe-42b1-a71d-7f9f020e12fb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:16.453250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1315] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:16.458278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3007c9-3b1e-4190-983c-5710679f804b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:16.458657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3007c9-3b1e-4190-983c-5710679f804b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:16.458847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3007c9-3b1e-4190-983c-5710679f804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:16.524926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b3007c9-3b1e-4190-983c-5710679f804b None None] MFA Rules not processed for user `98034ca1d05b4b12b4b8ee5fe315e059`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:16.526787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b3007c9-3b1e-4190-983c-5710679f804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:16.577418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:16.622660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1316] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:16.662851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a4aec-1bbe-42b1-a71d-7f9f020e12fb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:16.664700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c92a4aec-1bbe-42b1-a71d-7f9f020e12fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:16.675481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:16.777332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a4aec-1bbe-42b1-a71d-7f9f020e12fb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:16.854832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:16.855714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:16.856002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:16.856277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:16.857903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:16.924324 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:16.972276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:16.973729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471ac2e-d7c3-4eec-b9d8-9b4871848ab6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:16.992644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:19.652800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9665-eebd-489e-98d2-815f8376c2e1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:19.653371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9665-eebd-489e-98d2-815f8376c2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:19.653371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9665-eebd-489e-98d2-815f8376c2e1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:19.654727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1318] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:19.659902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c925c5-5ae8-4681-af50-e26219b3b081 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:19.660295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c925c5-5ae8-4681-af50-e26219b3b081 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:19.660470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c925c5-5ae8-4681-af50-e26219b3b081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:19.727787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-12c925c5-5ae8-4681-af50-e26219b3b081 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:19.729723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12c925c5-5ae8-4681-af50-e26219b3b081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:19.829450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1319] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:19.867486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9665-eebd-489e-98d2-815f8376c2e1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:19.869017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39bb9665-eebd-489e-98d2-815f8376c2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:19.972969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9665-eebd-489e-98d2-815f8376c2e1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:20.046477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:20.047100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:20.047409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:20.047693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:20.049315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:20.056190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:20.161184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:20.162638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97fedf-6d22-41f0-a1e2-59db7fcf1e09 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:20.181691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:22.951623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d09f07-72dd-4a3d-b9e4-72bf792b4af0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:22.952258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d09f07-72dd-4a3d-b9e4-72bf792b4af0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:22.952258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d09f07-72dd-4a3d-b9e4-72bf792b4af0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:22.953555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1321] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:22.958801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa6aed-4f0d-4957-8f87-6b0199525fa9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:22.959225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa6aed-4f0d-4957-8f87-6b0199525fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:22.959397 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa6aed-4f0d-4957-8f87-6b0199525fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:23.030481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41fa6aed-4f0d-4957-8f87-6b0199525fa9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:23.031750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41fa6aed-4f0d-4957-8f87-6b0199525fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:23.094433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:23.134099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1322] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:23.185537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d256bc7-1b8e-4a91-bcaa-e995930a0657 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:23.185833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d256bc7-1b8e-4a91-bcaa-e995930a0657 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:23.186119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d256bc7-1b8e-4a91-bcaa-e995930a0657 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:23.187567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1323] 10.222.0.49 () {54 vars in 965 bytes} [Fri Jul 29 05:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:23.194588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e348e-bc8f-4a6c-af9a-24f1aad0e3fe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:23.195331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e348e-bc8f-4a6c-af9a-24f1aad0e3fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:23.195644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e348e-bc8f-4a6c-af9a-24f1aad0e3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:23.283093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd0e348e-bc8f-4a6c-af9a-24f1aad0e3fe None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:23.284297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd0e348e-bc8f-4a6c-af9a-24f1aad0e3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:23.377315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1324] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Jul 29 05:12:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:23.387313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d256bc7-1b8e-4a91-bcaa-e995930a0657 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:23.388910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d256bc7-1b8e-4a91-bcaa-e995930a0657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:23.410269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:23.499743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d256bc7-1b8e-4a91-bcaa-e995930a0657 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:23.577157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:23.577707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:23.577991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:23.578269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:23.579950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:23.660965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:23.705624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:23.707085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f7349-3901-400b-80b7-bd414794bfc4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:23.726863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:26.046954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d468cc-789c-4da9-8cfc-06acc329dc63 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:26.047527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d468cc-789c-4da9-8cfc-06acc329dc63 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:26.047642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d468cc-789c-4da9-8cfc-06acc329dc63 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:26.049173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1326] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:26.054687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e06e6-f4be-4755-99be-f21e1401852b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:26.055258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e06e6-f4be-4755-99be-f21e1401852b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:26.055258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e06e6-f4be-4755-99be-f21e1401852b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:26.129217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d77e06e6-f4be-4755-99be-f21e1401852b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:26.130625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d77e06e6-f4be-4755-99be-f21e1401852b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:26.241458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:26.281311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d468cc-789c-4da9-8cfc-06acc329dc63 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:26.282667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5d468cc-789c-4da9-8cfc-06acc329dc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:26.397075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:26.455486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d468cc-789c-4da9-8cfc-06acc329dc63 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:26.536223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:26.536769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:26.537059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:26.537333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:26.539223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:26.677158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:26.680560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:26.683276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89db61-4568-4015-9205-0d965e9b1891 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:26.703110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:28.851979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbbbf3-4e6a-4fcd-8773-09ed28eceb2e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:28.852524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbbbf3-4e6a-4fcd-8773-09ed28eceb2e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:28.852524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbbbf3-4e6a-4fcd-8773-09ed28eceb2e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:28.853982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1329] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:28.859456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9010de-796c-4879-b0d6-c585eef44c18 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:28.859934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9010de-796c-4879-b0d6-c585eef44c18 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:28.860087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9010de-796c-4879-b0d6-c585eef44c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:28.929451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9b9010de-796c-4879-b0d6-c585eef44c18 None None] MFA Rules not processed for user `98034ca1d05b4b12b4b8ee5fe315e059`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:28.930664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b9010de-796c-4879-b0d6-c585eef44c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:29.057609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:29.096978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbbbf3-4e6a-4fcd-8773-09ed28eceb2e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:29.098603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-11bbbbf3-4e6a-4fcd-8773-09ed28eceb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:29.134995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:29.228115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbbbf3-4e6a-4fcd-8773-09ed28eceb2e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:29.457894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:29.458522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:29.458813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:29.459084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:29.460748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:29.549631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:29.600117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:29.601623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b4eb3-e9fa-4456-966b-e5d5ed84e082 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:29.623687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:31.949139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa8d0a-23b3-4cd6-8baf-f7fb52103d3b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:31.949731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa8d0a-23b3-4cd6-8baf-f7fb52103d3b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:31.949731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa8d0a-23b3-4cd6-8baf-f7fb52103d3b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:31.951044 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1332] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:12:31.956531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb8fb0-c2a8-4311-9818-ecae9d8856d2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:31.956987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb8fb0-c2a8-4311-9818-ecae9d8856d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:31.957159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb8fb0-c2a8-4311-9818-ecae9d8856d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:32.028966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f1cb8fb0-c2a8-4311-9818-ecae9d8856d2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:32.030209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f1cb8fb0-c2a8-4311-9818-ecae9d8856d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:32.131518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:32.168898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa8d0a-23b3-4cd6-8baf-f7fb52103d3b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:32.170544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2aa8d0a-23b3-4cd6-8baf-f7fb52103d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:32.248201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:32.284433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa8d0a-23b3-4cd6-8baf-f7fb52103d3b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:32.362896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:32.363555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:32.363839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:32.364115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:32.365770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:32.476531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:32.477939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7efa7f-1f71-4046-bd83-2035acf3b427 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:32.487998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:32.498248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:34.615730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544523e4-e7da-4bd2-a7e9-0d3d4a50d5eb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:34.616552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544523e4-e7da-4bd2-a7e9-0d3d4a50d5eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:34.616552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544523e4-e7da-4bd2-a7e9-0d3d4a50d5eb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:34.618202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1335] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:34.624582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956f6e-5f5e-4cea-9170-9e657ab193bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:34.625212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956f6e-5f5e-4cea-9170-9e657ab193bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:34.625212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956f6e-5f5e-4cea-9170-9e657ab193bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:34.712917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5956f6e-5f5e-4cea-9170-9e657ab193bb None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:34.714198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5956f6e-5f5e-4cea-9170-9e657ab193bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:34.817682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:34.858113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544523e4-e7da-4bd2-a7e9-0d3d4a50d5eb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:34.859804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-544523e4-e7da-4bd2-a7e9-0d3d4a50d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:34.905686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:34.975607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544523e4-e7da-4bd2-a7e9-0d3d4a50d5eb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:35.060126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:35.060762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:35.061052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:35.061336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:35.063012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:35.160572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:35.190292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:35.191794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e81fd-627d-48a0-a122-e43a11246f82 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:35.213261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:37.984523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a054ca9-555c-4307-bd5e-09eabe0e4250 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:37.985655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a054ca9-555c-4307-bd5e-09eabe0e4250 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:37.985655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a054ca9-555c-4307-bd5e-09eabe0e4250 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:37.986454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1338] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:37.992719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c494c7-2162-455d-b10a-c2478692bf64 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:37.993575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c494c7-2162-455d-b10a-c2478692bf64 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:37.993937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c494c7-2162-455d-b10a-c2478692bf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:38.099829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f9c494c7-2162-455d-b10a-c2478692bf64 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:38.102275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9c494c7-2162-455d-b10a-c2478692bf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:38.245034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:38.289051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a054ca9-555c-4307-bd5e-09eabe0e4250 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:38.291535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a054ca9-555c-4307-bd5e-09eabe0e4250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:38.416923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:38.456467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a054ca9-555c-4307-bd5e-09eabe0e4250 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:38.545283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:38.545921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:38.546400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:38.546889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:38.550201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:38.693517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:38.695059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0463856-24be-4eed-9da9-5538a3cf0efc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:38.720413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:43.362008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dee8f-eade-4bf3-a6e4-486b6790196d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:43.362635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dee8f-eade-4bf3-a6e4-486b6790196d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:43.362635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dee8f-eade-4bf3-a6e4-486b6790196d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:43.364026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1341] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:43.370909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf74a-430b-46ea-9000-05262094e17e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:43.371565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf74a-430b-46ea-9000-05262094e17e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:43.371565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf74a-430b-46ea-9000-05262094e17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:43.377146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:43.480708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-892cf74a-430b-46ea-9000-05262094e17e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:43.481891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-892cf74a-430b-46ea-9000-05262094e17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:43.604264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:43.660319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dee8f-eade-4bf3-a6e4-486b6790196d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:43.661869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe3dee8f-eade-4bf3-a6e4-486b6790196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:43.717484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:43.793623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dee8f-eade-4bf3-a6e4-486b6790196d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:43.895330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:43.898327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:43.898327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:43.898327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:43.901062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:44.014145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:44.034519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:44.036129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b1e4d-10ec-48e3-a7e7-92e64842beb6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:44.060044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:12:47.188025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506a6d-d4e0-4b97-80ba-c0ba3a7b6124 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:47.188555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506a6d-d4e0-4b97-80ba-c0ba3a7b6124 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:47.188649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506a6d-d4e0-4b97-80ba-c0ba3a7b6124 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:47.189963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1344] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:47.195462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14154e-42b1-4d24-9d44-8d764b91b682 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:47.196122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14154e-42b1-4d24-9d44-8d764b91b682 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:47.196122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14154e-42b1-4d24-9d44-8d764b91b682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:47.266316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8c14154e-42b1-4d24-9d44-8d764b91b682 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:47.267611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c14154e-42b1-4d24-9d44-8d764b91b682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:47.384753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:47.426766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506a6d-d4e0-4b97-80ba-c0ba3a7b6124 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:47.428362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd506a6d-d4e0-4b97-80ba-c0ba3a7b6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:47.523283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:47.542816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506a6d-d4e0-4b97-80ba-c0ba3a7b6124 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:47.619314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:47.619981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:47.620271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:47.620554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:47.622222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:47.736642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:47.738109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad05a7-2e15-4161-8ecc-56ca0738510d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:47.756916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:52.181216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7efbf2-fb7e-483b-90b9-14c36867cb0c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:52.181763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7efbf2-fb7e-483b-90b9-14c36867cb0c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:52.181763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7efbf2-fb7e-483b-90b9-14c36867cb0c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:52.183147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1347] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:52.188648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5728e3-af2e-4d34-8964-4b8a50253b03 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:52.189109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5728e3-af2e-4d34-8964-4b8a50253b03 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:52.189209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5728e3-af2e-4d34-8964-4b8a50253b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:52.208379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:52.271393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9b5728e3-af2e-4d34-8964-4b8a50253b03 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:52.272649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b5728e3-af2e-4d34-8964-4b8a50253b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:52.392993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:12:52.435859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7efbf2-fb7e-483b-90b9-14c36867cb0c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:12:52.437437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d7efbf2-fb7e-483b-90b9-14c36867cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:52.498291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:52.556614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7efbf2-fb7e-483b-90b9-14c36867cb0c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:12:52.637783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:12:52.638421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:52.638707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:52.638983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:52.640649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:52.747743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:12:52.751845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:12:52.753320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f7c78-7232-4f06-8ab4-41c3b888a6e9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:12:52.771206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:12:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:12:57.945107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea92645-ae44-4f02-8de1-85ff246d4a2c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:57.945107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea92645-ae44-4f02-8de1-85ff246d4a2c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:57.946370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea92645-ae44-4f02-8de1-85ff246d4a2c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:57.948907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1350] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:12:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:12:57.954351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a300f-471d-4a46-817e-cc922cf60502 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:12:57.954819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a300f-471d-4a46-817e-cc922cf60502 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:12:57.954928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a300f-471d-4a46-817e-cc922cf60502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:12:58.061014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a22a300f-471d-4a46-817e-cc922cf60502 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:12:58.062223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a22a300f-471d-4a46-817e-cc922cf60502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:12:58.289080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:12:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:02.390353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=59 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:02.971163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1b17f-86dc-45ce-8111-4590511bfa4d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:02.971669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1b17f-86dc-45ce-8111-4590511bfa4d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:02.972374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1b17f-86dc-45ce-8111-4590511bfa4d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:02.974699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1352] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:13:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:02.981264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe7005-79e8-4e39-b3bb-bcd0dbf73768 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:02.981801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe7005-79e8-4e39-b3bb-bcd0dbf73768 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:02.981801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe7005-79e8-4e39-b3bb-bcd0dbf73768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:03.054272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:03.084332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bbbe7005-79e8-4e39-b3bb-bcd0dbf73768 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:03.085564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bbbe7005-79e8-4e39-b3bb-bcd0dbf73768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:03.235975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:04.545958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a6d3-1441-492d-994f-e5f38c0d8154 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:04.546485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a6d3-1441-492d-994f-e5f38c0d8154 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:04.546485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a6d3-1441-492d-994f-e5f38c0d8154 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:04.547853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1354] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:04.553297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e2f25-9505-4b93-950f-c9da6be395ef None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:04.553745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e2f25-9505-4b93-950f-c9da6be395ef None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:04.553876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e2f25-9505-4b93-950f-c9da6be395ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:04.633607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb0e2f25-9505-4b93-950f-c9da6be395ef None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:04.634930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb0e2f25-9505-4b93-950f-c9da6be395ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:04.677896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:04.752227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:13:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:08.112554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67412c15-c01e-45e3-b7bc-e6f3affc840d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:08.113168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67412c15-c01e-45e3-b7bc-e6f3affc840d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:08.113168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67412c15-c01e-45e3-b7bc-e6f3affc840d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:08.114774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1356] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:08.125502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f615cd9-523e-413e-9e19-a01a78978864 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:08.126027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f615cd9-523e-413e-9e19-a01a78978864 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:08.126122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f615cd9-523e-413e-9e19-a01a78978864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:08.214235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f615cd9-523e-413e-9e19-a01a78978864 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:08.215674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f615cd9-523e-413e-9e19-a01a78978864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:08.354060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:08.412383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0daec5-2837-4e98-a413-8177b5d63e0b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:08.412920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0daec5-2837-4e98-a413-8177b5d63e0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:08.413709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0daec5-2837-4e98-a413-8177b5d63e0b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:08.416157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1358] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 05:13:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:08.423933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee249-2be2-47f4-a4d0-a80d7598d057 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:08.424344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee249-2be2-47f4-a4d0-a80d7598d057 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:08.424725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee249-2be2-47f4-a4d0-a80d7598d057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:08.442986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:08.525087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-125ee249-2be2-47f4-a4d0-a80d7598d057 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:08.526487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-125ee249-2be2-47f4-a4d0-a80d7598d057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:08.657208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 05:13:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:08.666097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0daec5-2837-4e98-a413-8177b5d63e0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:08.667778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5c0daec5-2837-4e98-a413-8177b5d63e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:08.843571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0daec5-2837-4e98-a413-8177b5d63e0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:08.931035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:08.973708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:08.974348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:08.974634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:08.974913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:08.976594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:09.074297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:09.075971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788e667-e082-43fb-bcd0-a8b7e7ea3105 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:09.098736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:13:11.452445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab087fe-a379-496f-b044-9f3eda58c1ca None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:11.453236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab087fe-a379-496f-b044-9f3eda58c1ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:11.453236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab087fe-a379-496f-b044-9f3eda58c1ca None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:11.455722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1361] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:11.459723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e2d5-e34e-47c4-8bbc-e849e11b4e2d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:11.460416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e2d5-e34e-47c4-8bbc-e849e11b4e2d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:11.460416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e2d5-e34e-47c4-8bbc-e849e11b4e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:11.542312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6010e2d5-e34e-47c4-8bbc-e849e11b4e2d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:11.543565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6010e2d5-e34e-47c4-8bbc-e849e11b4e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:11.590192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:11.679408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:11.727977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab087fe-a379-496f-b044-9f3eda58c1ca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:11.729680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ab087fe-a379-496f-b044-9f3eda58c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:11.812322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:11.929070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab087fe-a379-496f-b044-9f3eda58c1ca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:12.065850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:12.066496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:12.066856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:12.067138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:12.068927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:12.097000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:12.175430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:12.177069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a40308-7c83-4fc2-b375-81a3ead7c6cd service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:12.199934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:14.823844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71499b91-fd2a-434b-8418-536415022aed None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:14.824699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71499b91-fd2a-434b-8418-536415022aed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:14.824699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71499b91-fd2a-434b-8418-536415022aed None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:14.825976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1364] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:13:14.831594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ec203-6b39-4950-97e6-8d3ba9c56499 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:14.832102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ec203-6b39-4950-97e6-8d3ba9c56499 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:14.832210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ec203-6b39-4950-97e6-8d3ba9c56499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:14.911673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4d8ec203-6b39-4950-97e6-8d3ba9c56499 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:14.912986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d8ec203-6b39-4950-97e6-8d3ba9c56499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:15.053415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:15.059485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:15.114169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf656a8-66e8-4058-9b27-2d7683b50920 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:15.114920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf656a8-66e8-4058-9b27-2d7683b50920 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:15.115209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf656a8-66e8-4058-9b27-2d7683b50920 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:15.116716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1366] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 05:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:15.123375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9d7a-f277-4f48-b0a3-c99bf0ae2459 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:15.123746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9d7a-f277-4f48-b0a3-c99bf0ae2459 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:15.124134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9d7a-f277-4f48-b0a3-c99bf0ae2459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:15.231867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ebb9d7a-f277-4f48-b0a3-c99bf0ae2459 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:15.233118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ebb9d7a-f277-4f48-b0a3-c99bf0ae2459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:15.386597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 05:13:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:15.394779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf656a8-66e8-4058-9b27-2d7683b50920 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:15.396347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dcf656a8-66e8-4058-9b27-2d7683b50920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:15.567484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf656a8-66e8-4058-9b27-2d7683b50920 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:15.592726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:15.714189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:15.714833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:15.715122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:15.715438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:15.717163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:15.834550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:15.836225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98905386-2301-4a24-a792-9daf6b90385e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:15.858120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:16.675815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4085-6a14-41f8-9f60-1f82cfaa7b4b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:16.676386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4085-6a14-41f8-9f60-1f82cfaa7b4b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:16.676732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4085-6a14-41f8-9f60-1f82cfaa7b4b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:16.678984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1369] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:16.687325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51647651-e3a2-4a98-b797-bc8e9d7656a9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:16.687835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51647651-e3a2-4a98-b797-bc8e9d7656a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:16.688302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51647651-e3a2-4a98-b797-bc8e9d7656a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:16.740703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:16.796107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-51647651-e3a2-4a98-b797-bc8e9d7656a9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:16.797907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51647651-e3a2-4a98-b797-bc8e9d7656a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:16.921511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:17.236530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330199a2-bb67-4cbf-943b-1e516db4ff3f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:17.236827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330199a2-bb67-4cbf-943b-1e516db4ff3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:17.237112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330199a2-bb67-4cbf-943b-1e516db4ff3f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:17.238649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1371] 10.222.0.49 () {54 vars in 964 bytes} [Fri Jul 29 05:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:17.244457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea429c-0e1f-42dc-8069-202b20c8020e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:17.245073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea429c-0e1f-42dc-8069-202b20c8020e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:17.245817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea429c-0e1f-42dc-8069-202b20c8020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:17.334744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6ea429c-0e1f-42dc-8069-202b20c8020e None None] MFA Rules not processed for user `0a47f6ceb2a146b092ebeb72e8679335`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:17.335994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6ea429c-0e1f-42dc-8069-202b20c8020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:17.386489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:17.419382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Jul 29 05:13:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:17.426831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330199a2-bb67-4cbf-943b-1e516db4ff3f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:17.428706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-330199a2-bb67-4cbf-943b-1e516db4ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:17.492083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:17.558965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330199a2-bb67-4cbf-943b-1e516db4ff3f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:17.647553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:17.648352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:17.649787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:17.652722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:17.652722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:17.800805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:17.830675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:17.833633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8fa53-f41b-4414-96e8-813f1e3777ed service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:17.882331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:18.058886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09d9d8-99c4-4f24-ba47-2b5d59c98d48 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:18.059450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09d9d8-99c4-4f24-ba47-2b5d59c98d48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:18.059620 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09d9d8-99c4-4f24-ba47-2b5d59c98d48 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:18.061135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1374] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:13:18.067425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e88e-827a-476b-8db0-962c454f1f77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:18.067869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e88e-827a-476b-8db0-962c454f1f77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:18.068267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e88e-827a-476b-8db0-962c454f1f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:18.157449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f65e88e-827a-476b-8db0-962c454f1f77 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:18.158802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f65e88e-827a-476b-8db0-962c454f1f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:18.264762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:18.311171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09d9d8-99c4-4f24-ba47-2b5d59c98d48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:18.312747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a09d9d8-99c4-4f24-ba47-2b5d59c98d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:18.465764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09d9d8-99c4-4f24-ba47-2b5d59c98d48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:18.519378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:18.588398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:18.589050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:18.589341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:18.589625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:18.591266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:18.685260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:18.686869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c329c1c-bb36-4227-a7c5-2e97f3227a64 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:18.705510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:20.814975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ca06f-1868-49bb-bd23-738333412783 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:20.815566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ca06f-1868-49bb-bd23-738333412783 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:20.815566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ca06f-1868-49bb-bd23-738333412783 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:20.816948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1377] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:20.822506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc66093-ea97-4d25-8ac2-ead286ec1fe9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:20.822988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc66093-ea97-4d25-8ac2-ead286ec1fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:20.823104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc66093-ea97-4d25-8ac2-ead286ec1fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:20.903927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7dc66093-ea97-4d25-8ac2-ead286ec1fe9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:20.905197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7dc66093-ea97-4d25-8ac2-ead286ec1fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:20.929354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:21.033307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:21.078532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ca06f-1868-49bb-bd23-738333412783 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:21.080083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-210ca06f-1868-49bb-bd23-738333412783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:21.145967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:21.244877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ca06f-1868-49bb-bd23-738333412783 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:21.411271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:21.412238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:21.412524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:21.412809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:21.414967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:21.441487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:21.541067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:21.542922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a15118-1e45-44c1-a799-257d7c1306c9 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:21.564933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:23.687230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d13c-4136-49d1-b711-2b72f9ff7103 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:23.688018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d13c-4136-49d1-b711-2b72f9ff7103 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:23.688018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d13c-4136-49d1-b711-2b72f9ff7103 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:23.689407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1380] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:13:23.694926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634ee7-5c89-43af-8403-e06c7e99b0b3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:23.695851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634ee7-5c89-43af-8403-e06c7e99b0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:23.695851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634ee7-5c89-43af-8403-e06c7e99b0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:23.772676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ae634ee7-5c89-43af-8403-e06c7e99b0b3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:23.773909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae634ee7-5c89-43af-8403-e06c7e99b0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:23.880174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:23.889997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:23.937004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d13c-4136-49d1-b711-2b72f9ff7103 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:23.938591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2823d13c-4136-49d1-b711-2b72f9ff7103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:24.114067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:24.132373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d13c-4136-49d1-b711-2b72f9ff7103 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:24.290453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:24.291267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:24.291596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:24.291864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:24.293475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:24.395971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:24.397580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074702df-1abb-47b9-8739-77d4b4c6abb8 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:24.409382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:24.420771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:26.584913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5be9b-2f9e-4fe2-a694-e3830481733e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:26.585420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5be9b-2f9e-4fe2-a694-e3830481733e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:26.585420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5be9b-2f9e-4fe2-a694-e3830481733e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:26.587460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1383] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:26.593361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71071af5-d39f-4f9f-b052-56ca9e257e36 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:26.593846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71071af5-d39f-4f9f-b052-56ca9e257e36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:26.593939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71071af5-d39f-4f9f-b052-56ca9e257e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:26.685927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-71071af5-d39f-4f9f-b052-56ca9e257e36 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:26.687732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71071af5-d39f-4f9f-b052-56ca9e257e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:26.810948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:26.853622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5be9b-2f9e-4fe2-a694-e3830481733e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:26.855201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1f5be9b-2f9e-4fe2-a694-e3830481733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:27.054990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5be9b-2f9e-4fe2-a694-e3830481733e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:27.129170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:27.185111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:27.186097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:27.186626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:27.187028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:27.189416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:27.341544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:27.343379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9f2e1-f80c-4388-9c55-c447bc1eb11b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:27.367124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:30.002570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4686-362e-4624-8fb6-0439b2eb1ed3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:30.003396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4686-362e-4624-8fb6-0439b2eb1ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:30.003396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4686-362e-4624-8fb6-0439b2eb1ed3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:30.005210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1386] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:30.013434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7207545-ae51-41c5-84a6-77cd9fee0329 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:30.014195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7207545-ae51-41c5-84a6-77cd9fee0329 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:30.014195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7207545-ae51-41c5-84a6-77cd9fee0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:30.089988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b7207545-ae51-41c5-84a6-77cd9fee0329 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:30.091230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b7207545-ae51-41c5-84a6-77cd9fee0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:30.133301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:30.208532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:30.258148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df4686-362e-4624-8fb6-0439b2eb1ed3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:30.260369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05df4686-362e-4624-8fb6-0439b2eb1ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:30.350892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:30.466244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df4686-362e-4624-8fb6-0439b2eb1ed3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:30.614919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:30.615769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:30.616125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:30.616489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:30.618768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:30.689583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:30.778798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:30.781285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3a7e7-f2e4-434c-a758-34fe8641e836 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:30.809518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:33.190827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de872ce2-b580-4b2e-b740-02af30c3eacc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.191508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de872ce2-b580-4b2e-b740-02af30c3eacc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.192018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de872ce2-b580-4b2e-b740-02af30c3eacc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.194283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1389] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:33.202520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.203887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.204385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.286646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732fe7-c18d-48ed-8c61-2970fe3ef599 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.287133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732fe7-c18d-48ed-8c61-2970fe3ef599 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.287569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732fe7-c18d-48ed-8c61-2970fe3ef599 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.289128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1390] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:13:33.289343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:33.292250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:33.296342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119982d-7577-46e2-b426-dbb8e2a941fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.296780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119982d-7577-46e2-b426-dbb8e2a941fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.297164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119982d-7577-46e2-b426-dbb8e2a941fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.377463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5119982d-7577-46e2-b426-dbb8e2a941fd None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:33.378730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5119982d-7577-46e2-b426-dbb8e2a941fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:33.425098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1391] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:33.471274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:33.472629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:33.488820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:33.495015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1392] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:33.556035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f67e0-e4a4-4cbc-81e2-57ad960fb704 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.556335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f67e0-e4a4-4cbc-81e2-57ad960fb704 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.556900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f67e0-e4a4-4cbc-81e2-57ad960fb704 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.558395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1393] 10.222.0.49 () {54 vars in 966 bytes} [Fri Jul 29 05:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:33.563902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.564198 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.564476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.585082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:33.616669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de831a-c533-4cdc-b625-7fda373acaef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:33.681164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] MFA Rules not processed for user `7c858ed2d88a43afaf34e83155629c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:33.682621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:33.774563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1394] 10.222.0.49 () {58 vars in 1063 bytes} [Fri Jul 29 05:13:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:33.782875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:33.787193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:33.787915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.788205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.788493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.791452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:33.794332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc6d3b-f1a6-4879-a2d3-250d93c97cb1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:33.802743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:33.912178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:33.912843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:33.913134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:33.913416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:33.916021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:33.925706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:33.927447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b079e6-45ce-49ed-b265-c613e2f620d6 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:33.950590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1395] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:13:33.968225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:34.091141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:34.092770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb783c14-1dd4-4884-8c95-6c8f0a45ebf1 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:34.115894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:13:35.020688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d307487-52e5-41b6-bdc0-fbe099a2b550 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:35.021388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d307487-52e5-41b6-bdc0-fbe099a2b550 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:35.021388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d307487-52e5-41b6-bdc0-fbe099a2b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:35.103244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2d307487-52e5-41b6-bdc0-fbe099a2b550 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:35.104438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2d307487-52e5-41b6-bdc0-fbe099a2b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:35.217896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:35.223668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1397] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:13:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:35.273476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bde5-9623-45cb-8032-9b093940a3a3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:35.274053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bde5-9623-45cb-8032-9b093940a3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:35.275604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bde5-9623-45cb-8032-9b093940a3a3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:35.277190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1398] 10.222.0.49 () {54 vars in 943 bytes} [Fri Jul 29 05:13:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:13:35.284897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6d9fb-342b-4403-8adb-c288f411f1dd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:35.285397 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6d9fb-342b-4403-8adb-c288f411f1dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:35.285845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6d9fb-342b-4403-8adb-c288f411f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:35.379515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dda6d9fb-342b-4403-8adb-c288f411f1dd None None] MFA Rules not processed for user `1176e3df3023409b93375dc94ffb661c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:35.380666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dda6d9fb-342b-4403-8adb-c288f411f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:35.506302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1399] 10.222.0.49 () {58 vars in 1040 bytes} [Fri Jul 29 05:13:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:35.517776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bde5-9623-45cb-8032-9b093940a3a3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:35.519429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3628bde5-9623-45cb-8032-9b093940a3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:35.700894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:35.704421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bde5-9623-45cb-8032-9b093940a3a3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:35.845043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:35.845776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:35.846129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:35.846501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:35.848399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:35.954817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:35.956803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751abeac-bbd1-4593-a202-d3d0c148e4c3 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:35.967500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1400] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:13:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:13:36.137577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bf155-3a64-4201-912a-0acdefc83e91 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:36.138409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bf155-3a64-4201-912a-0acdefc83e91 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:36.138409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bf155-3a64-4201-912a-0acdefc83e91 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:36.140108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1401] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:36.148697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7595e-ca83-4f2e-8906-d3eaf52e3a9c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:36.149295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7595e-ca83-4f2e-8906-d3eaf52e3a9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:36.150313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7595e-ca83-4f2e-8906-d3eaf52e3a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:36.218816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:36.259474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c1e7595e-ca83-4f2e-8906-d3eaf52e3a9c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:36.260735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1e7595e-ca83-4f2e-8906-d3eaf52e3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:36.382193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1402] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:36.434269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bf155-3a64-4201-912a-0acdefc83e91 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:36.437259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f3bf155-3a64-4201-912a-0acdefc83e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:36.526029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:36.689838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bf155-3a64-4201-912a-0acdefc83e91 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:37.000187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:37.000950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:37.001407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:37.001707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:37.003523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:37.028631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:37.162923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:37.167692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd8aa2-d4c8-46e4-b5e0-54e4e70074cb service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:37.209890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:40.019396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e65b5-6ecb-46a9-9324-61e6b37888ed None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:40.020260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e65b5-6ecb-46a9-9324-61e6b37888ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:40.020260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e65b5-6ecb-46a9-9324-61e6b37888ed None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:40.021353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1404] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:13:40.028427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42409d2a-2ed6-40d3-818d-58c8ab72e478 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:40.029259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42409d2a-2ed6-40d3-818d-58c8ab72e478 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:40.029444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42409d2a-2ed6-40d3-818d-58c8ab72e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:40.183409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-42409d2a-2ed6-40d3-818d-58c8ab72e478 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:40.185704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42409d2a-2ed6-40d3-818d-58c8ab72e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:40.369579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:40.381246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:13:40.436248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e65b5-6ecb-46a9-9324-61e6b37888ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:40.439042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df8e65b5-6ecb-46a9-9324-61e6b37888ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:40.601500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:40.698630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e65b5-6ecb-46a9-9324-61e6b37888ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:40.887410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:40.888027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:40.888301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:40.888706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:40.890842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:41.015743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:41.051139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:41.054282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326653-71d6-4974-8c3f-f39fe4c0b776 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:41.095262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:13:43.341147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029be2dd-c036-41aa-bddf-99801cb91259 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:43.341715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029be2dd-c036-41aa-bddf-99801cb91259 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:43.341715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029be2dd-c036-41aa-bddf-99801cb91259 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:43.342980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1407] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:43.348624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e8047-99c1-4b45-a968-b264984362c1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:43.349593 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e8047-99c1-4b45-a968-b264984362c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:43.350113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e8047-99c1-4b45-a968-b264984362c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:43.444836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-be8e8047-99c1-4b45-a968-b264984362c1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:43.448969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-be8e8047-99c1-4b45-a968-b264984362c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:43.604731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:13:43.654741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029be2dd-c036-41aa-bddf-99801cb91259 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:13:43.656317 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-029be2dd-c036-41aa-bddf-99801cb91259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:43.878096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029be2dd-c036-41aa-bddf-99801cb91259 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:13:43.952918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:44.008066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:13:44.009085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:44.009375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:44.009672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:44.011521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:13:44.138180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:13:44.140012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dcd35-2ae0-4872-89ba-ace0475e9e81 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:13:44.161926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:13:53.588374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=79 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:54.159176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=64 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:13:59.810871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad0731-221a-428d-b365-ae3bdca42338 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:59.811454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad0731-221a-428d-b365-ae3bdca42338 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:59.811454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad0731-221a-428d-b365-ae3bdca42338 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:59.812855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1410] 10.222.0.49 () {54 vars in 918 bytes} [Fri Jul 29 05:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:13:59.818334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565edec-9a31-45fa-9df7-1573c188be31 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:13:59.818811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565edec-9a31-45fa-9df7-1573c188be31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:13:59.818918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565edec-9a31-45fa-9df7-1573c188be31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:13:59.900419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f565edec-9a31-45fa-9df7-1573c188be31 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:13:59.901675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f565edec-9a31-45fa-9df7-1573c188be31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:00.009553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jul 29 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:00.025408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f90c-f941-47b5-8359-356b0e3523d4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:00.025700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f90c-f941-47b5-8359-356b0e3523d4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:00.025984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f90c-f941-47b5-8359-356b0e3523d4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:00.027399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1412] 10.222.0.49 () {54 vars in 918 bytes} [Fri Jul 29 05:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:00.032606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe475-edc8-4e78-b008-179e5879d331 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:00.033030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe475-edc8-4e78-b008-179e5879d331 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:00.033361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe475-edc8-4e78-b008-179e5879d331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:00.115641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:00.119566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92afe475-edc8-4e78-b008-179e5879d331 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:00.121535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92afe475-edc8-4e78-b008-179e5879d331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:00.276662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jul 29 05:14:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:02.846615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9cc33-b213-492c-8acf-786fc6e381ca None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:02.847189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9cc33-b213-492c-8acf-786fc6e381ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:02.847189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9cc33-b213-492c-8acf-786fc6e381ca None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:02.848580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:02.854193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3f0e-81a5-4aa8-a15d-d20f6ff412f3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:02.854611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3f0e-81a5-4aa8-a15d-d20f6ff412f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:02.854759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3f0e-81a5-4aa8-a15d-d20f6ff412f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:02.924785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-32ab3f0e-81a5-4aa8-a15d-d20f6ff412f3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:02.926040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32ab3f0e-81a5-4aa8-a15d-d20f6ff412f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:02.982900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:03.027494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:03.071070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9cc33-b213-492c-8acf-786fc6e381ca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:03.073434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8cd9cc33-b213-492c-8acf-786fc6e381ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:03.222559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9cc33-b213-492c-8acf-786fc6e381ca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:03.296017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:03.341623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:03.342253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:03.342540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:03.342818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:03.344492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:03.435946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:03.437634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a30dc5-b25a-4ec0-a5eb-55d73204a8a8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:03.456493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:04.363126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368bd68-4024-47e4-bcf3-428cff43ed6d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:04.363719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368bd68-4024-47e4-bcf3-428cff43ed6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:04.363820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368bd68-4024-47e4-bcf3-428cff43ed6d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:04.365154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:04.371094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e9be5-9a32-4a49-a852-b741b6470442 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:04.371589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e9be5-9a32-4a49-a852-b741b6470442 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:04.371692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e9be5-9a32-4a49-a852-b741b6470442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:04.453566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a2e9be5-9a32-4a49-a852-b741b6470442 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:04.454977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a2e9be5-9a32-4a49-a852-b741b6470442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:04.484368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:04.558720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:04.717886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368bd68-4024-47e4-bcf3-428cff43ed6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:04.719381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f368bd68-4024-47e4-bcf3-428cff43ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:04.879692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368bd68-4024-47e4-bcf3-428cff43ed6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:04.922265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:05.001000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:05.001548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:05.001837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:05.002120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:05.003804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:05.114835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:05.116506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6770e-0cc4-491c-af5c-e40577f78596 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:05.137050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:06.058046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1afd9-976e-4a02-9ece-b53b2adf132a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:06.058644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1afd9-976e-4a02-9ece-b53b2adf132a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:06.058644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1afd9-976e-4a02-9ece-b53b2adf132a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:06.060138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:06.065271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fdba0-8ee9-4358-9dff-886693802b50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:06.065687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fdba0-8ee9-4358-9dff-886693802b50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:06.065844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fdba0-8ee9-4358-9dff-886693802b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:06.121922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:06.147115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b02fdba0-8ee9-4358-9dff-886693802b50 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:06.148388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b02fdba0-8ee9-4358-9dff-886693802b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:06.247207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:06.400655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1afd9-976e-4a02-9ece-b53b2adf132a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:06.402164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12b1afd9-976e-4a02-9ece-b53b2adf132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:06.550180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1afd9-976e-4a02-9ece-b53b2adf132a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:06.563208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:06.669141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:06.669675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:06.669958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:06.670233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:06.671890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:06.767326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:06.768970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f56ea-6caa-4af0-84dd-7b45eca47c63 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:06.787819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:07.676464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fddd9-7545-48bc-acc0-c9cf184594bd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:07.677041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fddd9-7545-48bc-acc0-c9cf184594bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:07.677041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fddd9-7545-48bc-acc0-c9cf184594bd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:07.678423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:07.684959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60831c-0666-4f61-a581-6442f484fafb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:07.685466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60831c-0666-4f61-a581-6442f484fafb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:07.685942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60831c-0666-4f61-a581-6442f484fafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:07.716083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:07.778000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6c60831c-0666-4f61-a581-6442f484fafb None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:07.779184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c60831c-0666-4f61-a581-6442f484fafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:07.882038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:08.035535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fddd9-7545-48bc-acc0-c9cf184594bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:08.037059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d03fddd9-7545-48bc-acc0-c9cf184594bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:08.165566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:08.203328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fddd9-7545-48bc-acc0-c9cf184594bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:08.319912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:08.320462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:08.320756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:08.321030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:08.324023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:08.404953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:08.419886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:08.421538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830086c-a400-42d7-8781-c5938bcfb079 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:08.440400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:09.331672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff74f5-4008-4aa9-b8a6-6b11a4d90278 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:09.332295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff74f5-4008-4aa9-b8a6-6b11a4d90278 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:09.332295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff74f5-4008-4aa9-b8a6-6b11a4d90278 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:09.333648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:09.339723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd69dd4-4a58-427b-9b08-35c87500bc30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:09.340616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd69dd4-4a58-427b-9b08-35c87500bc30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:09.340805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd69dd4-4a58-427b-9b08-35c87500bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:09.423583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fbd69dd4-4a58-427b-9b08-35c87500bc30 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:09.424823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fbd69dd4-4a58-427b-9b08-35c87500bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:09.524156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:14:09.676822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff74f5-4008-4aa9-b8a6-6b11a4d90278 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:09.678340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03ff74f5-4008-4aa9-b8a6-6b11a4d90278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:09.766497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:09.833887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff74f5-4008-4aa9-b8a6-6b11a4d90278 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:09.972072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:09.973192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:09.973607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:09.974065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:09.976085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:10.037290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:10.083332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:10.088476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfb5d3-08b9-45c6-a806-e3f332633113 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:10.116216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:10.973497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e731e-b2fe-4ba0-ae53-083b9050598b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:10.974168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e731e-b2fe-4ba0-ae53-083b9050598b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:10.974168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e731e-b2fe-4ba0-ae53-083b9050598b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:10.975577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:14:10.981111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2e77b-3382-4822-a966-cd59b0fd735f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:10.981616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2e77b-3382-4822-a966-cd59b0fd735f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:10.981729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2e77b-3382-4822-a966-cd59b0fd735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:11.064728 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2f2e77b-3382-4822-a966-cd59b0fd735f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:11.065953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2f2e77b-3382-4822-a966-cd59b0fd735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:11.164239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:11.323579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e731e-b2fe-4ba0-ae53-083b9050598b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:11.325024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-142e731e-b2fe-4ba0-ae53-083b9050598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:11.375557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:11.498529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e731e-b2fe-4ba0-ae53-083b9050598b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:11.647442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:11.648528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:11.649037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:11.650103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:11.653375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:11.666630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:11.758117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:11.759841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd250cc6-684e-4ae6-9795-a335fa085c80 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:11.785074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:12.624560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda5350-14e7-46a6-9488-cf6f62271d0e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:12.625102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda5350-14e7-46a6-9488-cf6f62271d0e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:12.625102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda5350-14e7-46a6-9488-cf6f62271d0e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:12.626443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:12.635968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e273e0f-5b0f-450a-a54b-8f64a30558e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:12.636556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e273e0f-5b0f-450a-a54b-8f64a30558e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:12.636556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e273e0f-5b0f-450a-a54b-8f64a30558e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:12.843753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5e273e0f-5b0f-450a-a54b-8f64a30558e4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:12.845011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e273e0f-5b0f-450a-a54b-8f64a30558e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:12.919617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:12.955492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:13.107107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda5350-14e7-46a6-9488-cf6f62271d0e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:13.108938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fda5350-14e7-46a6-9488-cf6f62271d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:13.128565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:13.267015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda5350-14e7-46a6-9488-cf6f62271d0e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:13.372031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:13.419142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:13.419903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:13.420187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:13.420463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:13.422277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:13.533543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:13.535087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e1998-5bc7-45af-9adb-101a3922adf8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:13.556534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:14.392560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabcaed-ffef-4b32-b26f-9a24ea28d211 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:14.393383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabcaed-ffef-4b32-b26f-9a24ea28d211 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:14.393383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabcaed-ffef-4b32-b26f-9a24ea28d211 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:14.394630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:14.400981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51c6b8-f59b-47cd-8f65-9e557f4cc836 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:14.401810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51c6b8-f59b-47cd-8f65-9e557f4cc836 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:14.401810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51c6b8-f59b-47cd-8f65-9e557f4cc836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:14.483768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5c51c6b8-f59b-47cd-8f65-9e557f4cc836 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:14.484908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c51c6b8-f59b-47cd-8f65-9e557f4cc836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:14.528599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:14.586862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:14:14.737057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabcaed-ffef-4b32-b26f-9a24ea28d211 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:14.738554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9dabcaed-ffef-4b32-b26f-9a24ea28d211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:14.894903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabcaed-ffef-4b32-b26f-9a24ea28d211 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:14.952615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:15.018250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:15.018797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:15.019084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:15.019420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:15.021045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:15.115480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:15.117111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abb18e-443c-44bd-92a1-6ec0d2a11e2d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:15.136993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:15.958550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6f295-cbea-45a2-a7c8-eefb8dacad71 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:15.959134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6f295-cbea-45a2-a7c8-eefb8dacad71 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:15.959237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6f295-cbea-45a2-a7c8-eefb8dacad71 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:15.960680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:15.967463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8740116-bb10-4b4d-810a-80191254d51e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:15.967957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8740116-bb10-4b4d-810a-80191254d51e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:15.968065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8740116-bb10-4b4d-810a-80191254d51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:16.061625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e8740116-bb10-4b4d-810a-80191254d51e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:16.062860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e8740116-bb10-4b4d-810a-80191254d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:16.065012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:16.172971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:16.330371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6f295-cbea-45a2-a7c8-eefb8dacad71 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:16.332201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3b6f295-cbea-45a2-a7c8-eefb8dacad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:16.496465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6f295-cbea-45a2-a7c8-eefb8dacad71 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:16.529117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:16.625591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:16.626177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:16.626462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:16.626745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:16.628492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:16.730748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:16.732424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785011de-626c-4fef-95a8-223cbdbb8f77 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:16.751825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:18.726525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a569d-7342-4e8a-a185-42ec241bfca4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:18.727115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a569d-7342-4e8a-a185-42ec241bfca4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:18.727115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a569d-7342-4e8a-a185-42ec241bfca4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:18.728751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:18.734456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59254e-e7ee-422c-9697-b7c7a2897565 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:18.734954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59254e-e7ee-422c-9697-b7c7a2897565 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:18.735073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59254e-e7ee-422c-9697-b7c7a2897565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:18.782495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:18.811864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7c59254e-e7ee-422c-9697-b7c7a2897565 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:18.813109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7c59254e-e7ee-422c-9697-b7c7a2897565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:18.924777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:18.966133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a569d-7342-4e8a-a185-42ec241bfca4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:18.967728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-433a569d-7342-4e8a-a185-42ec241bfca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:19.083046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a569d-7342-4e8a-a185-42ec241bfca4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:19.116365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:19.162430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:19.163058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:19.163342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:19.163674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:19.165367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:19.304285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:19.307724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100447f-946e-4dc9-b4ee-63449ef2ee89 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:19.343342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:21.716911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e976df-5a5e-494d-82f4-44c37db41e04 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:21.717450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e976df-5a5e-494d-82f4-44c37db41e04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:21.717544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e976df-5a5e-494d-82f4-44c37db41e04 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:21.718896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:21.724929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbde42-acff-4c5d-b613-c528ea7dff4b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:21.724929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbde42-acff-4c5d-b613-c528ea7dff4b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:21.725604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbde42-acff-4c5d-b613-c528ea7dff4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:21.799918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bfbbde42-acff-4c5d-b613-c528ea7dff4b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:21.801146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bfbbde42-acff-4c5d-b613-c528ea7dff4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:21.815416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:21.905337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:21.943545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e976df-5a5e-494d-82f4-44c37db41e04 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:21.945150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97e976df-5a5e-494d-82f4-44c37db41e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:22.051954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e976df-5a5e-494d-82f4-44c37db41e04 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:22.064240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:22.128999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:22.129628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:22.129908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:22.130183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:22.131837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:22.273384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:22.275457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0f304-6917-44a1-bbd3-b65e792ab5f6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:22.296495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:24.727602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80beafe-2071-41b7-afe8-ddb76e5ba596 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:24.728839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80beafe-2071-41b7-afe8-ddb76e5ba596 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:24.729209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80beafe-2071-41b7-afe8-ddb76e5ba596 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:24.731739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:24.739973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1897ee-0390-425b-8130-21d8a72103ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:24.740573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1897ee-0390-425b-8130-21d8a72103ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:24.740573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1897ee-0390-425b-8130-21d8a72103ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:24.794890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:24.818443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1a1897ee-0390-425b-8130-21d8a72103ee None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:24.819865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1a1897ee-0390-425b-8130-21d8a72103ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:24.931933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:14:24.985709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80beafe-2071-41b7-afe8-ddb76e5ba596 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:24.987326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c80beafe-2071-41b7-afe8-ddb76e5ba596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:25.104369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80beafe-2071-41b7-afe8-ddb76e5ba596 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:25.152278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:25.191748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:25.192435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:25.192771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:25.193156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:25.198919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:25.326383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:25.327825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2c8b-f9c8-4f16-b1a6-27d015c25902 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:25.349186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:27.661338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b9597-6d01-4ae7-a8be-d7509083a8f7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:27.662197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b9597-6d01-4ae7-a8be-d7509083a8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:27.662385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b9597-6d01-4ae7-a8be-d7509083a8f7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:27.664036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:14:27.671476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904a4a-a907-477d-ab8b-8970464fe11c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:27.672612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904a4a-a907-477d-ab8b-8970464fe11c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:27.673122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904a4a-a907-477d-ab8b-8970464fe11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:27.752588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f904a4a-a907-477d-ab8b-8970464fe11c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:27.753914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f904a4a-a907-477d-ab8b-8970464fe11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:27.777987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:27.866905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:14:27.908339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b9597-6d01-4ae7-a8be-d7509083a8f7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:27.909989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c93b9597-6d01-4ae7-a8be-d7509083a8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:28.065103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b9597-6d01-4ae7-a8be-d7509083a8f7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:28.088260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:28.154840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:28.155519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:28.155860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:28.156175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:28.158409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:28.300498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:28.301992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11d2e2-03b1-4a2b-8d6a-f49d830d154f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:28.323959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:30.604732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9def-f08c-4a59-ad6c-63e308153a6c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:30.605354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9def-f08c-4a59-ad6c-63e308153a6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:30.605354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9def-f08c-4a59-ad6c-63e308153a6c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:30.606696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:30.613231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19f338-5a2f-44dd-a851-c121d4d99867 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:30.613766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19f338-5a2f-44dd-a851-c121d4d99867 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:30.613904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19f338-5a2f-44dd-a851-c121d4d99867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:30.686660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:30.690308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b19f338-5a2f-44dd-a851-c121d4d99867 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:30.692437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b19f338-5a2f-44dd-a851-c121d4d99867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:30.802785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:14:30.842951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9def-f08c-4a59-ad6c-63e308153a6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:30.844542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea6d9def-f08c-4a59-ad6c-63e308153a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:30.965389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9def-f08c-4a59-ad6c-63e308153a6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:31.018889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:31.046194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:31.046830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:31.047119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:31.047445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:31.049110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:31.171862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:31.173631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d105e-7ea6-4c49-9d3f-b2b9aaa49980 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:31.195463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:33.330498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c067c-fc91-4af3-826b-9a1ad47d6251 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:33.331074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c067c-fc91-4af3-826b-9a1ad47d6251 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:33.331074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c067c-fc91-4af3-826b-9a1ad47d6251 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:33.332493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:33.338004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725be49-05ac-4a43-914f-1e6497d60e14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:33.338445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725be49-05ac-4a43-914f-1e6497d60e14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:33.338638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725be49-05ac-4a43-914f-1e6497d60e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:33.406761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1725be49-05ac-4a43-914f-1e6497d60e14 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:33.408034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1725be49-05ac-4a43-914f-1e6497d60e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:33.438785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:33.514803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:33.554779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c067c-fc91-4af3-826b-9a1ad47d6251 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:33.557303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f0c067c-fc91-4af3-826b-9a1ad47d6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:33.677529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c067c-fc91-4af3-826b-9a1ad47d6251 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:33.712814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:33.764494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:33.765127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:33.765411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:33.765688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:33.767383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:33.895179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:33.896701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dba0d7-506c-4d31-9aae-5bfb1d18cc9e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:33.918163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:36.054809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1054d1-9ec5-4099-928f-16b0604f91d0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:36.055428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1054d1-9ec5-4099-928f-16b0604f91d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:36.055428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1054d1-9ec5-4099-928f-16b0604f91d0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:36.056822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:14:36.062815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bb24c-64e8-4bfb-9e12-2fcd69edaa2a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:36.063390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bb24c-64e8-4bfb-9e12-2fcd69edaa2a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:36.063390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bb24c-64e8-4bfb-9e12-2fcd69edaa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:36.142974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a46bb24c-64e8-4bfb-9e12-2fcd69edaa2a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:36.144301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a46bb24c-64e8-4bfb-9e12-2fcd69edaa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:36.153419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:36.253376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:36.293157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1054d1-9ec5-4099-928f-16b0604f91d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:36.294703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea1054d1-9ec5-4099-928f-16b0604f91d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:36.412436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1054d1-9ec5-4099-928f-16b0604f91d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:36.471824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:36.492206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:36.492836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:36.493119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:36.493465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:36.495100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:36.609137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:36.610594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1639b-7ecc-4497-a434-170b272fdea8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:36.630016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:38.789019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be433623-fd3e-4d8d-a7a3-acc187a48591 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:38.789962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be433623-fd3e-4d8d-a7a3-acc187a48591 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:38.789962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be433623-fd3e-4d8d-a7a3-acc187a48591 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:38.791863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:14:38.798584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803961dc-69a3-480d-bb09-f50ab0b2b3d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:38.799200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803961dc-69a3-480d-bb09-f50ab0b2b3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:38.799200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803961dc-69a3-480d-bb09-f50ab0b2b3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:38.879731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-803961dc-69a3-480d-bb09-f50ab0b2b3d1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:38.880992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-803961dc-69a3-480d-bb09-f50ab0b2b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:38.925528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:38.988686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:14:39.028126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be433623-fd3e-4d8d-a7a3-acc187a48591 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:39.029751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-be433623-fd3e-4d8d-a7a3-acc187a48591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:39.143983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be433623-fd3e-4d8d-a7a3-acc187a48591 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:39.190234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:39.230402 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:39.231051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:39.231341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:39.231644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:39.233389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:39.355329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:39.356853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256002d-98bf-4bdb-8635-412d49cf1304 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:39.378644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:41.474452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef216e-b212-4f6e-a1e8-296f6207af1e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:41.474989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef216e-b212-4f6e-a1e8-296f6207af1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:41.474989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef216e-b212-4f6e-a1e8-296f6207af1e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:41.476327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:14:41.482578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474f311-451e-4f87-995a-574ca2869b8c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:41.483059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474f311-451e-4f87-995a-574ca2869b8c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:41.483161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474f311-451e-4f87-995a-574ca2869b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:41.560065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d474f311-451e-4f87-995a-574ca2869b8c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:41.561310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d474f311-451e-4f87-995a-574ca2869b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:41.581206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:41.670496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:41.724244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef216e-b212-4f6e-a1e8-296f6207af1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:41.725968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52ef216e-b212-4f6e-a1e8-296f6207af1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:41.845518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef216e-b212-4f6e-a1e8-296f6207af1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:41.919129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:41.926286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:41.926932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:41.927211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:41.927517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:41.929158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:42.049394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:42.050789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315666e0-0f65-486e-bd1d-d734c037d71b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:42.072249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:44.194579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8f36-6119-4528-95a4-539265210294 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:44.195465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8f36-6119-4528-95a4-539265210294 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:44.195656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8f36-6119-4528-95a4-539265210294 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:44.197721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:14:44.204960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383501ce-e14f-42bc-a253-bca439ae853c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:44.205396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383501ce-e14f-42bc-a253-bca439ae853c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:44.205664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383501ce-e14f-42bc-a253-bca439ae853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:44.282706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-383501ce-e14f-42bc-a253-bca439ae853c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:14:44.283961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-383501ce-e14f-42bc-a253-bca439ae853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:44.342705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:44.398643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:14:44.436937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8f36-6119-4528-95a4-539265210294 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:14:44.438502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89cd8f36-6119-4528-95a4-539265210294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:44.557476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8f36-6119-4528-95a4-539265210294 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:14:44.617758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:44.648592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:14:44.649235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:14:44.650064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:14:44.650342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:14:44.652311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:14:44.771222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:14:44.772717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786b22e-6855-410d-8cd7-31722fd73b06 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:14:44.793481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:14:54.396770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=22 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:14:54.789845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=66 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:05.492181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b604-c714-4da8-998f-75185497f8c5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:05.493028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b604-c714-4da8-998f-75185497f8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:05.493028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b604-c714-4da8-998f-75185497f8c5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:05.494729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1471] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:15:05.500721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f04af-8902-4e9b-add9-267cb49f784b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:05.501279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f04af-8902-4e9b-add9-267cb49f784b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:05.501279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f04af-8902-4e9b-add9-267cb49f784b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:05.582613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e5f04af-8902-4e9b-add9-267cb49f784b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:05.583903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e5f04af-8902-4e9b-add9-267cb49f784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:05.716642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:05.811267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0fb7-ffdb-4369-9462-91cdba0e2f56 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:05.811597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0fb7-ffdb-4369-9462-91cdba0e2f56 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:05.811883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0fb7-ffdb-4369-9462-91cdba0e2f56 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:05.813261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1473] 10.222.0.49 () {54 vars in 966 bytes} [Fri Jul 29 05:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:15:05.819972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782fb27-b2e9-427f-8f3b-30af761c14cf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:05.820491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782fb27-b2e9-427f-8f3b-30af761c14cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:05.821023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782fb27-b2e9-427f-8f3b-30af761c14cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:05.923760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8782fb27-b2e9-427f-8f3b-30af761c14cf None None] MFA Rules not processed for user `7c858ed2d88a43afaf34e83155629c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:05.925038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8782fb27-b2e9-427f-8f3b-30af761c14cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:05.952068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:06.014795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1474] 10.222.0.49 () {58 vars in 1063 bytes} [Fri Jul 29 05:15:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:06.022942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0fb7-ffdb-4369-9462-91cdba0e2f56 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:06.024734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1bcc0fb7-ffdb-4369-9462-91cdba0e2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:06.151653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0fb7-ffdb-4369-9462-91cdba0e2f56 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:06.240214 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:06.240996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:06.241385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:06.241760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:06.244373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:06.313771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:06.401928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:06.404521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39da879-4922-496a-8a33-00ff1332d906 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:06.439138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:15:08.671018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeace6-6426-40d4-88dd-bd052360c7c7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:08.671674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeace6-6426-40d4-88dd-bd052360c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:08.671674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeace6-6426-40d4-88dd-bd052360c7c7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:08.673057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1476] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:15:08.678962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015171d-0d79-4cf1-a72c-7a54e9ff8574 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:08.679498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015171d-0d79-4cf1-a72c-7a54e9ff8574 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:08.679601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015171d-0d79-4cf1-a72c-7a54e9ff8574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:08.784964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f015171d-0d79-4cf1-a72c-7a54e9ff8574 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:08.786629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f015171d-0d79-4cf1-a72c-7a54e9ff8574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:08.909128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:15:08.933638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeace6-6426-40d4-88dd-bd052360c7c7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:08.936321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1beeace6-6426-40d4-88dd-bd052360c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:09.085451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeace6-6426-40d4-88dd-bd052360c7c7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:09.151478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:09.204618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:09.205311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:09.205638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:09.205962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:09.206716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:09.208634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:15:09.208634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:15:09.208634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:15:09.210027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba296f-e985-49fb-9a05-09ce13b576a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:09.232088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 05:15:08 2022] GET /identity/v3/projects => generated 2553 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:15:11.388270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58ac22-1ffd-4a5c-94fa-f9b58bad075d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:11.388962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58ac22-1ffd-4a5c-94fa-f9b58bad075d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:11.388962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58ac22-1ffd-4a5c-94fa-f9b58bad075d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:11.390207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1479] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:15:11.395637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062b97c-a6a5-4f8e-8215-20e463374319 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:11.396172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062b97c-a6a5-4f8e-8215-20e463374319 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:11.396290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062b97c-a6a5-4f8e-8215-20e463374319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:11.467060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7062b97c-a6a5-4f8e-8215-20e463374319 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:11.468248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7062b97c-a6a5-4f8e-8215-20e463374319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:11.568147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:11.617230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58ac22-1ffd-4a5c-94fa-f9b58bad075d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:11.619783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e58ac22-1ffd-4a5c-94fa-f9b58bad075d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:11.630523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:11.798451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58ac22-1ffd-4a5c-94fa-f9b58bad075d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:11.890351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:11.936884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:11.937996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:11.938529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:11.939054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:11.942454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:12.044619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:12.046134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba818c-6f71-439e-b623-b6d585ab80bd service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:12.065315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:15:14.328678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f71897-83da-421b-9c1e-fc33bc6fc40c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:14.329269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f71897-83da-421b-9c1e-fc33bc6fc40c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:14.329269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f71897-83da-421b-9c1e-fc33bc6fc40c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:14.330661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1482] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:15:14.337207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e24e4-9ec0-4f74-a96b-5338bd0c19b1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:14.337805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e24e4-9ec0-4f74-a96b-5338bd0c19b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:14.338451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e24e4-9ec0-4f74-a96b-5338bd0c19b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:14.406118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2f1e24e4-9ec0-4f74-a96b-5338bd0c19b1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:14.407370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f1e24e4-9ec0-4f74-a96b-5338bd0c19b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:14.456657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:14.509688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:14.546216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f71897-83da-421b-9c1e-fc33bc6fc40c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:14.547815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e8f71897-83da-421b-9c1e-fc33bc6fc40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:14.579753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:14.670235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f71897-83da-421b-9c1e-fc33bc6fc40c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:14.752393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:14.752967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:14.753257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:14.753544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:14.755226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:14.848454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:14.886853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:14.888357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21c87f-ebf6-48ab-89f6-ac9920d438b5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:14.909459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:15:17.011983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc58c4-7a9f-4af6-b7e7-787e54534cfd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:17.012565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc58c4-7a9f-4af6-b7e7-787e54534cfd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:17.012565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc58c4-7a9f-4af6-b7e7-787e54534cfd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:17.013954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1485] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:15:17.019947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ea47c-e25f-4847-b226-05b9fdd38549 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:17.020394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ea47c-e25f-4847-b226-05b9fdd38549 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:17.020516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ea47c-e25f-4847-b226-05b9fdd38549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:17.094227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3f2ea47c-e25f-4847-b226-05b9fdd38549 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:17.095545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f2ea47c-e25f-4847-b226-05b9fdd38549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:17.211293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:17.250361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc58c4-7a9f-4af6-b7e7-787e54534cfd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:17.251979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16cc58c4-7a9f-4af6-b7e7-787e54534cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:17.372455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc58c4-7a9f-4af6-b7e7-787e54534cfd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:17.446798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:17.449028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:17.449749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:17.450246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:17.450593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:17.452647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:17.570737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:17.573491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f347b06-824b-4e52-94d3-834aad4f6182 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:17.594247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:15:19.815826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f1bff-00c8-43cc-a111-f0498345fe1b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:19.816391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f1bff-00c8-43cc-a111-f0498345fe1b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:19.816391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f1bff-00c8-43cc-a111-f0498345fe1b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:19.817788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1488] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:15:19.824414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1d0c-2347-432d-afdf-6451a137ad06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:19.824885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1d0c-2347-432d-afdf-6451a137ad06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:19.824992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1d0c-2347-432d-afdf-6451a137ad06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:19.893279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b9a1d0c-2347-432d-afdf-6451a137ad06 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:19.894498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b9a1d0c-2347-432d-afdf-6451a137ad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:19.953150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:19.999409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:20.041257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f1bff-00c8-43cc-a111-f0498345fe1b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:20.043069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-191f1bff-00c8-43cc-a111-f0498345fe1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:20.079882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:20.155056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f1bff-00c8-43cc-a111-f0498345fe1b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:20.230855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:20.231433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:20.231858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:20.231985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:20.233620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:20.314616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:20.345585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:20.347040 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a358cc9-4916-4a0e-a181-bc3ab58934ab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:20.366959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:15:22.789361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cef9f-25a7-4445-81e5-3f6e8f972e5c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:22.789973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cef9f-25a7-4445-81e5-3f6e8f972e5c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:22.789973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cef9f-25a7-4445-81e5-3f6e8f972e5c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:22.791315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1491] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 05:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:15:22.797425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76f5a-dc27-4f5e-a9f0-ed40ac72b67d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:22.797963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76f5a-dc27-4f5e-a9f0-ed40ac72b67d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:22.797963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76f5a-dc27-4f5e-a9f0-ed40ac72b67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:22.870031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7fa76f5a-dc27-4f5e-a9f0-ed40ac72b67d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:15:22.871261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fa76f5a-dc27-4f5e-a9f0-ed40ac72b67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:22.983245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 05:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:15:23.024064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cef9f-25a7-4445-81e5-3f6e8f972e5c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:15:23.025841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-677cef9f-25a7-4445-81e5-3f6e8f972e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:23.140662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cef9f-25a7-4445-81e5-3f6e8f972e5c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:15:23.215716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:15:23.216355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:15:23.216643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:15:23.216920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:15:23.218534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:15:23.228845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:23.330975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:15:23.332454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e8ddc-f799-451d-aec6-d5b198746a24 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:15:23.351141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:15:32.981426 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=82 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:15:33.348904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=50 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:17:22.074876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.075557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.075557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.166365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:17:22.167636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:22.278320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1494] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:17:22.287074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.287699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.287699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.311755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:17:22.313028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:22.322379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1495] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:17:22.327368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.339095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad834b-97c1-42b5-8931-9e2066ed95d6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.344244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.344889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.345170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.345446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.346192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:17:22.346500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:22.347565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:22.347565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:17:22.347565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:17:22.348471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:22.361994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 763/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:17:22.370385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.380208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac7261-29f8-4a68-ba71-0bef3f355554 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.385802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.386439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.386725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.387009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.387725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:22.389522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:22.389522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:17:22.389522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:17:22.390737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:22.430741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:17:22 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:17:22.434918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.445542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a2b60-7225-4ebd-88b1-2820a139f9ef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.497579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:17:22.505390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.506038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.506328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.507113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.507839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:22.508891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:22.508891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:17:22.508891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:17:22.510120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:22.536022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 764/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:17:22 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:17:22.540527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.551848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8a458-e68a-47af-b0f1-facaef0f9bee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.556946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.557580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.557867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.558143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.558841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:22.559897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:22.559897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:17:22.559897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:17:22.560787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:22.574952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:17:22.579580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.590830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e060a-3788-49b9-9ebe-7f7e189f6b19 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.595644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.596275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.596559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.596840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.597540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:22.598563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:22.598563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:17:22.598563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:17:22.599493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09585a0c-c4d9-4c68-ac32-6dc6ec9bc5ad admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:22.614017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 765/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:17:22.618269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.628449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa14a-5868-44b4-ad8f-851e1176bfe8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.633641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.634319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.634599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.634875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] PATH_INFO: `/v3/projects/b313bdf29812427db498369fe28dfff3/users/be10ec288b934e58aacfead25df2223f/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.650946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b313bdf29812427db498369fe28dfff3, user_id=be10ec288b934e58aacfead25df2223f, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:22.652023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:22.652023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:17:22.652023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:17:22.653964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02956dc5-fa31-48ec-8463-86053cd69a92 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:22.667889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:17:22 2022] PUT /identity/v3/projects/b313bdf29812427db498369fe28dfff3/users/be10ec288b934e58aacfead25df2223f/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:17:22.680611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b7eb-3a3b-456f-a1f2-777f4cc19fb5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.680968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b7eb-3a3b-456f-a1f2-777f4cc19fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.681205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b7eb-3a3b-456f-a1f2-777f4cc19fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.721693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bff5b7eb-3a3b-456f-a1f2-777f4cc19fb5 None None] MFA Rules not processed for user `be10ec288b934e58aacfead25df2223f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:17:22.723329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bff5b7eb-3a3b-456f-a1f2-777f4cc19fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:22.804740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 766/1502] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:17:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:17:22.818056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91335360-ecdb-4e59-9f65-c78398f86e20 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.818340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91335360-ecdb-4e59-9f65-c78398f86e20 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.818645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91335360-ecdb-4e59-9f65-c78398f86e20 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.820799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1503] 10.222.0.49 () {54 vars in 964 bytes} [Fri Jul 29 05:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:17:22.825925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1654b-4a38-4ad9-94ce-ffad86b90c48 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.826224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1654b-4a38-4ad9-94ce-ffad86b90c48 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.826509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1654b-4a38-4ad9-94ce-ffad86b90c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:22.888570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b4b1654b-4a38-4ad9-94ce-ffad86b90c48 None None] MFA Rules not processed for user `0a47f6ceb2a146b092ebeb72e8679335`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:17:22.889791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4b1654b-4a38-4ad9-94ce-ffad86b90c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:22.972246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 767/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Jul 29 05:17:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:17:22.980469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91335360-ecdb-4e59-9f65-c78398f86e20 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:22.992223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91335360-ecdb-4e59-9f65-c78398f86e20 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:22.995880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:22.996508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:22.996796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:22.997073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:23.006248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:23.008794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:23.019106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:17:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:17:23.066076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049001b-b703-4dd2-817b-a9e92540ad49 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:23.066370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049001b-b703-4dd2-817b-a9e92540ad49 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:23.066658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049001b-b703-4dd2-817b-a9e92540ad49 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:23.067968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 768/1506] 10.222.0.49 () {52 vars in 830 bytes} [Fri Jul 29 05:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:17:23.072311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:23.135135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874be56f-f651-430b-abe5-bce9a8eb0f6e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:23.139980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:23.140591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:23.140880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:23.141161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] PATH_INFO: `/v3/users/be10ec288b934e58aacfead25df2223f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:23.142369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be10ec288b934e58aacfead25df2223f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:23.143434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:23.143434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:17:23.143434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:17:23.144674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:23.219866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be10ec288b934e58aacfead25df2223f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.220661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be10ec288b934e58aacfead25df2223f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.236759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be10ec288b934e58aacfead25df2223f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.237642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be10ec288b934e58aacfead25df2223f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.259056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be10ec288b934e58aacfead25df2223f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.263336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:17:23.277954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:17:23 2022] DELETE /identity/v3/users/be10ec288b934e58aacfead25df2223f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:17:23.282562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:23.283110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:23.283110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:23.305110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:17:23.306348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:23.316068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:17:23.370386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 769/1508] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:17:23.380790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:23.382358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:23.465201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545394b5-6bb0-4c94-b338-9b98e4306c2a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:23.468874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630da10-d5d6-4d0a-9e59-b7d721301862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:23.469507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630da10-d5d6-4d0a-9e59-b7d721301862 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:23.469798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630da10-d5d6-4d0a-9e59-b7d721301862 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:23.470088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630da10-d5d6-4d0a-9e59-b7d721301862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:23.480274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630da10-d5d6-4d0a-9e59-b7d721301862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:23.481745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630da10-d5d6-4d0a-9e59-b7d721301862 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:23.492584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:17:23.873180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:17:23.874864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:17:23.887931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04065b-588d-4bec-a8a5-ed1ac5033eef None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:17:23.892486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:17:23.893112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:17:23.893846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:17:23.893846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] PATH_INFO: `/v3/projects/b313bdf29812427db498369fe28dfff3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:17:23.894704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b313bdf29812427db498369fe28dfff3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:17:23.895746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:17:23.895746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:17:23.895746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:17:23.896910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:17:23.967187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b313bdf29812427db498369fe28dfff3', 'request_id': 'req-34765cca-1899-4c0a-8d84-a6a1706172cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.977078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b313bdf29812427db498369fe28dfff3', 'request_id': 'req-34765cca-1899-4c0a-8d84-a6a1706172cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:23.983877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b313bdf29812427db498369fe28dfff3', 'request_id': 'req-34765cca-1899-4c0a-8d84-a6a1706172cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:24.005455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] The token cache is being invalidate because project b313bdf29812427db498369fe28dfff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:17:24.005455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:24.005455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34765cca-1899-4c0a-8d84-a6a1706172cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:17:24.007711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 770/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:17:23 2022] DELETE /identity/v3/projects/b313bdf29812427db498369fe28dfff3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:17:33.487381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:17:33.999729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:19:00.761077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca07d9c-8100-40d2-9ee3-dfe69d071876 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:00.761707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca07d9c-8100-40d2-9ee3-dfe69d071876 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:00.761707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca07d9c-8100-40d2-9ee3-dfe69d071876 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:00.763079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:00.768335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629e38-f970-4198-9667-5ee225e5e47c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:00.768854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629e38-f970-4198-9667-5ee225e5e47c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:00.768972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629e38-f970-4198-9667-5ee225e5e47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:00.791631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b4629e38-f970-4198-9667-5ee225e5e47c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:00.793098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4629e38-f970-4198-9667-5ee225e5e47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:00.853111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 771/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:01.007426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca07d9c-8100-40d2-9ee3-dfe69d071876 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:01.009167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ca07d9c-8100-40d2-9ee3-dfe69d071876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:01.125321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca07d9c-8100-40d2-9ee3-dfe69d071876 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:01.131850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898da01-2ab8-4b8d-8d8a-20cdfdee6642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:01.132481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898da01-2ab8-4b8d-8d8a-20cdfdee6642 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:01.132769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898da01-2ab8-4b8d-8d8a-20cdfdee6642 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:01.133051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898da01-2ab8-4b8d-8d8a-20cdfdee6642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:01.141236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898da01-2ab8-4b8d-8d8a-20cdfdee6642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:01.142843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898da01-2ab8-4b8d-8d8a-20cdfdee6642 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:01.152849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:19:01.993407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a85a3-ac31-4c2b-a3c4-f1beb76810d9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:01.993971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a85a3-ac31-4c2b-a3c4-f1beb76810d9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:01.993971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a85a3-ac31-4c2b-a3c4-f1beb76810d9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:01.995492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 772/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:02.000761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de5d3d-f5f7-4af3-bb20-4da4dd878d7f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:02.001232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de5d3d-f5f7-4af3-bb20-4da4dd878d7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:02.001354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de5d3d-f5f7-4af3-bb20-4da4dd878d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:02.021300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17de5d3d-f5f7-4af3-bb20-4da4dd878d7f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:02.022454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17de5d3d-f5f7-4af3-bb20-4da4dd878d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:02.030274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:02.189261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a85a3-ac31-4c2b-a3c4-f1beb76810d9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:02.203282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a85a3-ac31-4c2b-a3c4-f1beb76810d9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:02.208827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e263b-ab43-4f43-adb8-71ad8fa2021a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:02.209439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e263b-ab43-4f43-adb8-71ad8fa2021a service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:02.209724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e263b-ab43-4f43-adb8-71ad8fa2021a service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:02.210001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e263b-ab43-4f43-adb8-71ad8fa2021a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:02.218367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e263b-ab43-4f43-adb8-71ad8fa2021a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:02.220013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e263b-ab43-4f43-adb8-71ad8fa2021a service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:02.230139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 773/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:03.154441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fce703-7131-4fef-b235-25bbede9be99 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:03.155002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fce703-7131-4fef-b235-25bbede9be99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:03.155101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fce703-7131-4fef-b235-25bbede9be99 None None] PATH_INFO: `/v3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:03.156419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1517] 10.222.0.60 () {54 vars in 925 bytes} [Fri Jul 29 05:19:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:19:03.163182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e37e-e899-492f-b198-d87528dcbe26 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:03.163624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e37e-e899-492f-b198-d87528dcbe26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:03.163826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e37e-e899-492f-b198-d87528dcbe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:03.215468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2310e37e-e899-492f-b198-d87528dcbe26 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:03.216695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2310e37e-e899-492f-b198-d87528dcbe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:03.290060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 774/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Jul 29 05:19:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:07.072577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115024e-93a4-45fb-bd90-bbf476fa952c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:07.073338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115024e-93a4-45fb-bd90-bbf476fa952c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:07.073338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115024e-93a4-45fb-bd90-bbf476fa952c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:07.074421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1519] 10.222.0.49 () {54 vars in 961 bytes} [Fri Jul 29 05:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:19:07.079926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70ae2-57a6-46d9-8f8b-e4542e80e21e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:07.080420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70ae2-57a6-46d9-8f8b-e4542e80e21e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:07.080420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70ae2-57a6-46d9-8f8b-e4542e80e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:07.099632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ef70ae2-57a6-46d9-8f8b-e4542e80e21e None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:07.100842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ef70ae2-57a6-46d9-8f8b-e4542e80e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:07.110258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 775/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Jul 29 05:19:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:07.118295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115024e-93a4-45fb-bd90-bbf476fa952c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:07.131946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115024e-93a4-45fb-bd90-bbf476fa952c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:07.139300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f903532-e04a-4f92-8a22-53dc50e2a620 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:07.139943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903532-e04a-4f92-8a22-53dc50e2a620 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:07.140233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903532-e04a-4f92-8a22-53dc50e2a620 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:07.140515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903532-e04a-4f92-8a22-53dc50e2a620 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:07.149374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f903532-e04a-4f92-8a22-53dc50e2a620 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:07.151085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f903532-e04a-4f92-8a22-53dc50e2a620 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:07.170389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:13.461925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874b164-337f-47a0-87ae-100f8562d7c0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:13.462492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874b164-337f-47a0-87ae-100f8562d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:13.462492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874b164-337f-47a0-87ae-100f8562d7c0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:13.463887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 776/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:13.468850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78b88-fc45-4720-a36f-77faf261f5d3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:13.469252 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78b88-fc45-4720-a36f-77faf261f5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:13.469420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78b88-fc45-4720-a36f-77faf261f5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:13.491080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-abd78b88-fc45-4720-a36f-77faf261f5d3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:13.493611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-abd78b88-fc45-4720-a36f-77faf261f5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:13.501483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:13.673212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874b164-337f-47a0-87ae-100f8562d7c0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:13.692855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874b164-337f-47a0-87ae-100f8562d7c0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:13.704069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21763629-0263-4b87-9a55-326489cc6bb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:13.704900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21763629-0263-4b87-9a55-326489cc6bb3 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:13.705214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21763629-0263-4b87-9a55-326489cc6bb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:13.705495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21763629-0263-4b87-9a55-326489cc6bb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:13.715977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21763629-0263-4b87-9a55-326489cc6bb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:13.717642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21763629-0263-4b87-9a55-326489cc6bb3 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:13.728266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 777/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:14.641716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c38d0-e4bf-4a8a-ba17-125820bff128 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:14.642548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c38d0-e4bf-4a8a-ba17-125820bff128 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:14.642548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c38d0-e4bf-4a8a-ba17-125820bff128 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:14.644660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 05:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:14.650453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5f980-5e4a-44ec-8da9-4c5bfb26d33d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:14.651539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5f980-5e4a-44ec-8da9-4c5bfb26d33d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:14.651539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5f980-5e4a-44ec-8da9-4c5bfb26d33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:14.672057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bbc5f980-5e4a-44ec-8da9-4c5bfb26d33d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:14.673274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bbc5f980-5e4a-44ec-8da9-4c5bfb26d33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:14.681462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 778/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 05:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:14.845574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c38d0-e4bf-4a8a-ba17-125820bff128 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:14.860023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c38d0-e4bf-4a8a-ba17-125820bff128 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:14.866191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d78c7a-3a1c-4ba1-b342-b3528c671ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:14.866822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78c7a-3a1c-4ba1-b342-b3528c671ed8 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:14.867288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78c7a-3a1c-4ba1-b342-b3528c671ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:14.867831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78c7a-3a1c-4ba1-b342-b3528c671ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:14.876332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d78c7a-3a1c-4ba1-b342-b3528c671ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:14.878172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d78c7a-3a1c-4ba1-b342-b3528c671ed8 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:14.889528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:18.818343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68301215-02ef-4a78-b996-69d4de551814 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:18.818921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68301215-02ef-4a78-b996-69d4de551814 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:18.818921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68301215-02ef-4a78-b996-69d4de551814 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:18.820864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 779/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:18.826320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb2fce-b0fe-4923-81b5-4b72bc691f60 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:18.826866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb2fce-b0fe-4923-81b5-4b72bc691f60 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:18.826968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb2fce-b0fe-4923-81b5-4b72bc691f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:18.847887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cefb2fce-b0fe-4923-81b5-4b72bc691f60 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:18.849120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cefb2fce-b0fe-4923-81b5-4b72bc691f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:18.857532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:19:18.932623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68301215-02ef-4a78-b996-69d4de551814 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:18.934376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68301215-02ef-4a78-b996-69d4de551814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:18.963023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:19:18.992688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68301215-02ef-4a78-b996-69d4de551814 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:18.996344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991620e7-8c23-46f4-a1ed-af15ecc3dece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:18.996984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991620e7-8c23-46f4-a1ed-af15ecc3dece service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:18.997269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991620e7-8c23-46f4-a1ed-af15ecc3dece service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:18.997551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991620e7-8c23-46f4-a1ed-af15ecc3dece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:19.006917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991620e7-8c23-46f4-a1ed-af15ecc3dece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:19.009459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991620e7-8c23-46f4-a1ed-af15ecc3dece service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:19.019018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 780/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:21.033385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547d943-42a4-4579-bd7d-8634ef5a7103 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:21.033958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547d943-42a4-4579-bd7d-8634ef5a7103 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:21.033958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547d943-42a4-4579-bd7d-8634ef5a7103 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:21.035322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:21.041062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b42ba-6c6b-4fd8-9eb8-43d97875d805 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:21.041540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b42ba-6c6b-4fd8-9eb8-43d97875d805 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:21.041664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b42ba-6c6b-4fd8-9eb8-43d97875d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:21.063150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5d8b42ba-6c6b-4fd8-9eb8-43d97875d805 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:21.064387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5d8b42ba-6c6b-4fd8-9eb8-43d97875d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:21.073115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 781/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:21.109623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547d943-42a4-4579-bd7d-8634ef5a7103 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:21.120989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547d943-42a4-4579-bd7d-8634ef5a7103 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:21.124671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bb697-ca69-4315-b536-08d0bf0abae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:21.125290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bb697-ca69-4315-b536-08d0bf0abae3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:21.125643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bb697-ca69-4315-b536-08d0bf0abae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:21.125930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bb697-ca69-4315-b536-08d0bf0abae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:21.136062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bb697-ca69-4315-b536-08d0bf0abae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:21.137515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bb697-ca69-4315-b536-08d0bf0abae3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:21.147000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:24.860075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26da09-a47d-4585-a44a-f182bdd51b7e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:24.860667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26da09-a47d-4585-a44a-f182bdd51b7e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:24.860667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26da09-a47d-4585-a44a-f182bdd51b7e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:24.862069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 782/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:24.867125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e0fdc-7f16-4c67-836c-3e5962df7815 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:24.867581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e0fdc-7f16-4c67-836c-3e5962df7815 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:24.867715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e0fdc-7f16-4c67-836c-3e5962df7815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:24.875325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=74 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:19:24.888270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d34e0fdc-7f16-4c67-836c-3e5962df7815 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:24.889472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d34e0fdc-7f16-4c67-836c-3e5962df7815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:24.897187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:25.205018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26da09-a47d-4585-a44a-f182bdd51b7e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:25.206696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef26da09-a47d-4585-a44a-f182bdd51b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:25.264549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26da09-a47d-4585-a44a-f182bdd51b7e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:25.268082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8787cd-35fd-4dc7-a99c-8a536ec857f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:25.268695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8787cd-35fd-4dc7-a99c-8a536ec857f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:25.268983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8787cd-35fd-4dc7-a99c-8a536ec857f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:25.269258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8787cd-35fd-4dc7-a99c-8a536ec857f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:25.278652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8787cd-35fd-4dc7-a99c-8a536ec857f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:25.280120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8787cd-35fd-4dc7-a99c-8a536ec857f7 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:25.289514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 783/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:27.330149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b161817-c8da-482f-9cd2-402206d61ce9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:27.330703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b161817-c8da-482f-9cd2-402206d61ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:27.330703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b161817-c8da-482f-9cd2-402206d61ce9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:27.332122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 05:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:19:27.337449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b52819-36ba-4c71-b8b7-7cc367cf69b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:27.337862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b52819-36ba-4c71-b8b7-7cc367cf69b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:27.338017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b52819-36ba-4c71-b8b7-7cc367cf69b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:27.357276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0b52819-36ba-4c71-b8b7-7cc367cf69b9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:19:27.358494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0b52819-36ba-4c71-b8b7-7cc367cf69b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:27.366828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 784/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 05:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:19:27.670988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b161817-c8da-482f-9cd2-402206d61ce9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:19:27.672725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b161817-c8da-482f-9cd2-402206d61ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:19:27.686329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b161817-c8da-482f-9cd2-402206d61ce9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:19:27.690002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31081c4d-ea8d-4e90-88dc-72459483aa9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:19:27.690634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31081c4d-ea8d-4e90-88dc-72459483aa9e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:19:27.690973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31081c4d-ea8d-4e90-88dc-72459483aa9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:19:27.691192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31081c4d-ea8d-4e90-88dc-72459483aa9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:19:27.700710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31081c4d-ea8d-4e90-88dc-72459483aa9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:19:27.702149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31081c4d-ea8d-4e90-88dc-72459483aa9e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:19:27.712099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:19:37.352462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=47 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:19:37.707357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:57.855290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7adc-5e38-45c1-b0e5-27c0afd90cf1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:57.855986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7adc-5e38-45c1-b0e5-27c0afd90cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:57.855986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7adc-5e38-45c1-b0e5-27c0afd90cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:57.880282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-901e7adc-5e38-45c1-b0e5-27c0afd90cf1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:57.881780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-901e7adc-5e38-45c1-b0e5-27c0afd90cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:57.894759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 785/1540] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:57.904965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4281f3a-15d3-4596-a6ee-529860611490 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:57.905922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4281f3a-15d3-4596-a6ee-529860611490 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:57.905922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4281f3a-15d3-4596-a6ee-529860611490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:57.921125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855fe1-1f81-4268-b83d-fc20d0b1bd02 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:57.921125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855fe1-1f81-4268-b83d-fc20d0b1bd02 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:57.921446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855fe1-1f81-4268-b83d-fc20d0b1bd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:57.944416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4281f3a-15d3-4596-a6ee-529860611490 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:57.945813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4281f3a-15d3-4596-a6ee-529860611490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:57.950568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-33855fe1-1f81-4268-b83d-fc20d0b1bd02 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:57.951780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33855fe1-1f81-4268-b83d-fc20d0b1bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:57.966729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 786/1541] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:20:57.967176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1542] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:57.968835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afac7b5-f681-48e6-b49e-9250f47f34bd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:57.969129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afac7b5-f681-48e6-b49e-9250f47f34bd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:57.969417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afac7b5-f681-48e6-b49e-9250f47f34bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:57.976531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:57.976855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:57.977106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:57.998256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6afac7b5-f681-48e6-b49e-9250f47f34bd None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.005095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6afac7b5-f681-48e6-b49e-9250f47f34bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.014063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 787/1543] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.015574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225e77-ced2-4b2e-b060-9da6b5be2e4d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.015868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225e77-ced2-4b2e-b060-9da6b5be2e4d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.016148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225e77-ced2-4b2e-b060-9da6b5be2e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.017183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.018426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.033663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1544] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:20:58.034587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.038475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-00225e77-ced2-4b2e-b060-9da6b5be2e4d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.039987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00225e77-ced2-4b2e-b060-9da6b5be2e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.045713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edffbb-216f-4f3d-b4e8-3598f3fa7c7d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.049584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 788/1545] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.050981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bdcb6-ca75-445c-8786-cd6eb23b5417 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.051102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.051377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bdcb6-ca75-445c-8786-cd6eb23b5417 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.051569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bdcb6-ca75-445c-8786-cd6eb23b5417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.051697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.051909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.052186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.053006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.053314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.054348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:20:58.054348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:20:58.054348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:20:58.055246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5780cd-6ab7-4b86-a638-8f93a7fad894 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.071423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1546] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.072802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.073096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.073376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.077640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-193bdcb6-ca75-445c-8786-cd6eb23b5417 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.078866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-193bdcb6-ca75-445c-8786-cd6eb23b5417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.091833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 789/1547] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.093281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8201e5-3bf0-4c9a-ab6f-4bff8390e977 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.093572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8201e5-3bf0-4c9a-ab6f-4bff8390e977 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.093846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8201e5-3bf0-4c9a-ab6f-4bff8390e977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.095422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.096650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.106810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.107454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.117349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c8201e5-3bf0-4c9a-ab6f-4bff8390e977 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.118549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c8201e5-3bf0-4c9a-ab6f-4bff8390e977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.118688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579d17b-3d48-4c29-bf7c-8ab68a766d78 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.123250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.123896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.124177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.124453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.125145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.125449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.126471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 790/1549] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.127055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.127748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca072f-f973-41f2-81bf-6b6b57beeefb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.128121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca072f-f973-41f2-81bf-6b6b57beeefb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.128375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca072f-f973-41f2-81bf-6b6b57beeefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.140996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1550] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.141678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.148931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-adca072f-f973-41f2-81bf-6b6b57beeefb None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.150136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adca072f-f973-41f2-81bf-6b6b57beeefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.152350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509290c-6565-44b5-9bc9-1ccfa2d154fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.157025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1363255-0e64-41c7-82c6-101da5208d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.157666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1363255-0e64-41c7-82c6-101da5208d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.157945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 791/1551] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.158061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1363255-0e64-41c7-82c6-101da5208d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.158245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1363255-0e64-41c7-82c6-101da5208d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.158868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1363255-0e64-41c7-82c6-101da5208d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.159304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634c414-adf4-4885-92ed-7b24b9e3f75e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.159619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634c414-adf4-4885-92ed-7b24b9e3f75e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.159896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634c414-adf4-4885-92ed-7b24b9e3f75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.160979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1363255-0e64-41c7-82c6-101da5208d1a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.182000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0634c414-adf4-4885-92ed-7b24b9e3f75e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.183375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0634c414-adf4-4885-92ed-7b24b9e3f75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.191893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 792/1552] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.192562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1553] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.193149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba80a4-333c-404f-aa17-760a9c23720b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.193646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba80a4-333c-404f-aa17-760a9c23720b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.193765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.193932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba80a4-333c-404f-aa17-760a9c23720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.194044 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.194248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.214528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.216277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.220319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ffba80a4-333c-404f-aa17-760a9c23720b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.221418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ffba80a4-333c-404f-aa17-760a9c23720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.277886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1554] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.285480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.285583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 793/1555] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.285583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.285583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.285583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.291165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47053e1c-8f41-4d7c-bcce-c479c64d33cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.295863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.296480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.296764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.297041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.297733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.298042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.299683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.305032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.306275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.311528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1556] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.312315 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.314439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 794/1557] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.315433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.324514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc945f2-e640-4991-8a0a-3e07861ab5fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.328108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1a9d9-2c42-45e9-83ae-1b54944e9932 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.329047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.329660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.329944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.330217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.330879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.332871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.333043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.333659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.334536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.334815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.335709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.336022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.337641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1c34-65d3-44f3-961d-bc2c57d14245 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.349749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 795/1558] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.351043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63369df5-5116-44b5-84a2-b4808d046b6a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.351340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63369df5-5116-44b5-84a2-b4808d046b6a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.351651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63369df5-5116-44b5-84a2-b4808d046b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.369906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1559] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.371534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.374542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-63369df5-5116-44b5-84a2-b4808d046b6a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.375872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-63369df5-5116-44b5-84a2-b4808d046b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.431708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:58.436390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc656931-230d-4873-aaf3-eb4d1000e1fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.440703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 796/1560] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.441977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.442277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.442685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.444223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.445164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.445479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.445761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.446443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.447503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:20:58.447503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:20:58.447503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:20:58.448763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.464384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:58.465656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:58.470849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1561] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.471706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.477130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 797/1562] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:58.477996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.484115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2cf02-9d8a-4766-84a9-f729512cdea2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.488913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.489530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.489703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2de45-0258-4d42-bc60-2e1f649e8c73 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.489851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.490128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.491313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.491657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.493328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.495281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.495913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.496200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.496479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.497170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.497480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.499067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.506091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1563] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.506893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.519169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 798/1564] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.519896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23c804-6807-4c6f-b236-e4b87a8d06e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.520126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.525187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.525809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.526136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.526422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.527093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.529101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.531321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac368-cf17-486b-bbe7-071a8ab9442a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.536267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.536954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.537738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.538019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.538686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.540017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:20:58.540017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:20:58.540017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:20:58.541256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.547075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:58.557523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1565] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.558753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.574606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 799/1566] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.574606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.627956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509da4d8-baab-4ffa-81ef-681efe3987a8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.633297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.633920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.634206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.634480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.635140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.635629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cf39f-93ef-48f8-97c0-81f2b61f7989 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.637106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.640392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.641051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.641399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.641723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.642427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.642740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.644381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.657394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 800/1567] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.660654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1568] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.660798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.660895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.673932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab084d-fc8d-400d-8713-b5aaf4103e3d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.674517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb992ed-1f3c-4ce7-9754-8e5afa40fd9a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.679253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.679895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.680015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.680181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.680459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.680572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.680826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.681108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.681219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:20:58.681434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.681803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.683061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.683421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.695438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 801/1569] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:58.696231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:20:58.696365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.700348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.707678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677f905-b325-4b10-a4ae-acde0efa9a0b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.711729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59b9d-893a-4ed7-b833-fdc558187f5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.713310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.713928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.714217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.714498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.715169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.716547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.717119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.717250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.717439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.717722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.718382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.720364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.748024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 802/1571] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.749082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.749819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1572] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.753802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.813786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2a61-7ce1-4164-ac2e-ded689db7083 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.816796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186387e-a634-4d2d-9803-094e943e8b47 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.819948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.820716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.821004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.821280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.821942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.822725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.823418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.823705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.823979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.824082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.824640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.826555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.838580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:58.857316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 803/1573] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.858158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1574] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.859030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.859152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.869851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63704aa-120c-42f2-96ae-7d7f637efe71 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.870533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd9ec2-39de-4887-bcaa-4674c8ddb322 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.874801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.875460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.875590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.875807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.876034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.876217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.876478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.876700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.876814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.877504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.878674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.879142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.890813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 804/1575] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:58.891867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.902526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49d0a3-7dcd-4fef-85ba-e07ba389105c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.910228 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:58.910731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.910857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.910936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.910936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.910936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.910936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.910936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.925665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 805/1577] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:58.925665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:58.933703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7a71b-c694-4a3c-88b2-eedf736f70f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.946351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:58.977879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fa985-685e-4256-815d-e838f79f9d30 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:58.982613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.983233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.983547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.983816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.984516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:58.986477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:58.995100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:58.995963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:58.996866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:58.997286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:58.998146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.000660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.021300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1578] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:59.021300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.027008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 806/1579] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:58 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:59.027956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.038586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708afb91-2f1a-4f5e-95cb-dceea9ff3cf5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.089312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc33f3-0c0d-4e24-8e6b-b7fc6c56f328 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.094040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.094670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.094951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.095232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.095928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.097851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.100842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.101663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.102062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.102484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.103508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.105344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.123152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 807/1580] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.123152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.126896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1581] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:59.127770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.134241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c49257-19c5-4978-a0d0-bdd576e6d7a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.137376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e8990-8c07-4f9d-bc03-d0236c01cc9a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.139535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.140159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.140442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.140718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.141537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.142367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.142992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.143135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.143307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.143629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.144379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.145975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.153910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 808/1582] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.154594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.157686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1583] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.158497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.166859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f03f6e-d5b4-4cd0-91ab-f1139600e6c4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.171021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001edcc3-109c-419c-897f-34d1fbae061e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.174151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.174831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.175126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.175254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.175471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] PATH_INFO: `/v3/projects/153fe5d9d64346f6b7419eba91948678/users/45ae88234e984149be8539757efa8733/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.175864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.176147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.176424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.177083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.179721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.192254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=153fe5d9d64346f6b7419eba91948678, user_id=45ae88234e984149be8539757efa8733, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.195330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.202483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 809/1584] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/users => generated 295 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:59.203522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.210218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1585] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:20:59 2022] PUT /identity/v3/projects/153fe5d9d64346f6b7419eba91948678/users/45ae88234e984149be8539757efa8733/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:20:59.211306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.214428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4279-05cf-46a8-8f3f-a0f611fed969 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.225751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:59.273186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.273817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.274111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.274399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.275064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.277074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.277956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775465a-674a-43b9-a71c-2499b2b4ea41 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.282844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.283511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.283805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.284107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.284801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.286547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.297817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1586] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.298655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.300416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 810/1587] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:20:59.301127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.306192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:59.309561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d51ff-c96d-4bd6-ae85-1162aaa1ef46 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.312991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ef2a1-6188-4564-82e1-1ad9b1e52e9a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.314857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.315505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.315794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.316109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.316801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.318806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.319123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.319400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.319695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.319976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.320689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.322279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.330169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.331019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.334781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 811/1589] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.335827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.344738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d64051-7825-4dc5-aa8d-a127351817cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.345921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f6b1c-f1c7-4635-af8e-3323252603f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.350745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.351409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.351702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.351814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.351983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.352434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.352675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.352776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.352984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] PATH_INFO: `/v3/projects/9d8baaf2e23546ee8ed1de278617c7d2/users/3c60b3a983f14f50b1b762a3954a1059/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.354293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.366094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.366765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.368654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8baaf2e23546ee8ed1de278617c7d2, user_id=3c60b3a983f14f50b1b762a3954a1059, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.369899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:20:59.369899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:20:59.369899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:20:59.371884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282f2a0-a4a6-4fe7-8004-c980140dfd5b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.376458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41e005-d256-4403-9588-2abd8df9f26b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.382749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.383378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.383748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.383953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.384698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.386273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.387454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 812/1591] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:20:59 2022] PUT /identity/v3/projects/9d8baaf2e23546ee8ed1de278617c7d2/users/3c60b3a983f14f50b1b762a3954a1059/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:20:59.391562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.391562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.391562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.397752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.398521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.408513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94398069-ca43-4fb3-a8e0-ebb55cea3371 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.411724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:59.413573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:59.466290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.466922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.467209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.467524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.468219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.469839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.478724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 813/1593] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:59.479685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.480464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.481135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.492547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebfa44-0875-4f7a-bbc7-dbedb8de99ba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.492756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209ff49-bb78-41ce-913d-3c1aa86fa293 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.497180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.497332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.497852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.497959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.498142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.498246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.498421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] PATH_INFO: `/v3/projects/908a005339464b1dacebb2fa00e0a14e/users/13be1d5917bc4a7ea41c83cce5c1e7f3/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.498516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.499176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.500836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.511132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 814/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.512062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.515685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908a005339464b1dacebb2fa00e0a14e, user_id=13be1d5917bc4a7ea41c83cce5c1e7f3, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.518448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.522790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a49d1-cb8e-4119-b7a3-c9df0b89f77e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.528171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8ac66-8215-434e-83ea-b1cc3759b54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.528793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ac66-8215-434e-83ea-b1cc3759b54e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.529076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ac66-8215-434e-83ea-b1cc3759b54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.529354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ac66-8215-434e-83ea-b1cc3759b54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.530085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8ac66-8215-434e-83ea-b1cc3759b54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.531709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8ac66-8215-434e-83ea-b1cc3759b54e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.533060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1596] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:20:59 2022] PUT /identity/v3/projects/908a005339464b1dacebb2fa00e0a14e/users/13be1d5917bc4a7ea41c83cce5c1e7f3/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:20:59.540725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.543821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 815/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.545265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.545557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.545833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.564612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:59.565811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:59.593259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:59.609073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2228f6-eb28-4626-964e-600370a65f24 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.613795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.614472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.614759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.615721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] PATH_INFO: `/v3/projects/22cd7e817de842b58375aefa6b0f1ff4/users/e20e6e74cb5d442ca07674bfa5627e41/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.627397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 816/1598] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:59.628452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.632878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22cd7e817de842b58375aefa6b0f1ff4, user_id=e20e6e74cb5d442ca07674bfa5627e41, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.635843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.639050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3c0e2-bc3b-4e7e-a695-98e3005a189f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.644541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.645068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.645356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.645644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.646388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.648306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.650414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1599] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:20:59 2022] PUT /identity/v3/projects/22cd7e817de842b58375aefa6b0f1ff4/users/e20e6e74cb5d442ca07674bfa5627e41/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:20:59.659418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.669979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 817/1600] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.670912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.711323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:59.722411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae92d3-3ca9-4200-88b4-bb99b8ffc366 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.728124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.728748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.729043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a207dcc-7b85-4c86-8900-13d51afeaf47 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.729194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.729338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.730038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.731735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.741978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1601] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:20:59.742769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.752962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c4749-9a67-4220-854c-6635a6f7ba13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.758243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.758905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.759194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.759527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] PATH_INFO: `/v3/projects/91858dac25b4484e95e7354ade64775d/users/47b7233b3a654d19a2182344fc5193a7/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.775410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91858dac25b4484e95e7354ade64775d, user_id=47b7233b3a654d19a2182344fc5193a7, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.778400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.783145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dfee2-7199-4fcf-9ed5-e81cda96595b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.783933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dfee2-7199-4fcf-9ed5-e81cda96595b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.784286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dfee2-7199-4fcf-9ed5-e81cda96595b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.784716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dfee2-7199-4fcf-9ed5-e81cda96595b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.793730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1602] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:20:59 2022] PUT /identity/v3/projects/91858dac25b4484e95e7354ade64775d/users/47b7233b3a654d19a2182344fc5193a7/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:20:59.794521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.799806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dfee2-7199-4fcf-9ed5-e81cda96595b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.801298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dfee2-7199-4fcf-9ed5-e81cda96595b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.811254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 818/1603] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:20:59.812689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426dc53-5461-4bed-bf02-3a370357ed09 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.812991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426dc53-5461-4bed-bf02-3a370357ed09 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.813275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426dc53-5461-4bed-bf02-3a370357ed09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.845126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6426dc53-5461-4bed-bf02-3a370357ed09 None None] MFA Rules not processed for user `13be1d5917bc4a7ea41c83cce5c1e7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:59.846740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6426dc53-5461-4bed-bf02-3a370357ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:20:59.871384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658fa6-588e-4374-b5f4-4c2a3eba9f94 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:20:59.878121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:20:59.878822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.879115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.879438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] PATH_INFO: `/v3/projects/c87396f5e6bf46f5b85720c8e365da81/users/720be309cbe14940873d774b06e91bf9/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.898323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87396f5e6bf46f5b85720c8e365da81, user_id=720be309cbe14940873d774b06e91bf9, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:20:59.901186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:20:59.917642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1604] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:20:59 2022] PUT /identity/v3/projects/c87396f5e6bf46f5b85720c8e365da81/users/720be309cbe14940873d774b06e91bf9/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:20:59.918710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:20:59.928135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 819/1605] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:20:59.929590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:20:59.930521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:20:59.930811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:20:59.963759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:20:59.964901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] MFA Rules not processed for user `e20e6e74cb5d442ca07674bfa5627e41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:20:59.966362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.000190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83dc10-2d10-4cc2-b5ca-7647d76e5eb8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.004355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.004909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.005197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.005534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.066768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 820/1606] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:20:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:00.066768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.080446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:00.096249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.097872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.110950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1607] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:00.111830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.121704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788177-78c9-49f7-9926-25d81b22a677 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.128143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdafee-2f6b-47e2-848b-7a1c957ab1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.128958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdafee-2f6b-47e2-848b-7a1c957ab1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.129345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdafee-2f6b-47e2-848b-7a1c957ab1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.129692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdafee-2f6b-47e2-848b-7a1c957ab1ab admin admin] PATH_INFO: `/v3/projects/92f16adc2357459aa0afb68db89031dd/users/c2db6ada97da4ee89f3ce8de98f2653a/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.143761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558d8be-fab4-4125-a234-8d9ddfc11c80 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.145972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdafee-2f6b-47e2-848b-7a1c957ab1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f16adc2357459aa0afb68db89031dd, user_id=c2db6ada97da4ee89f3ce8de98f2653a, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.149778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdafee-2f6b-47e2-848b-7a1c957ab1ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.150046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93b00f-cc1e-46b1-a3c6-7d577131d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.150840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93b00f-cc1e-46b1-a3c6-7d577131d554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.151195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93b00f-cc1e-46b1-a3c6-7d577131d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.151626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93b00f-cc1e-46b1-a3c6-7d577131d554 admin admin] PATH_INFO: `/v3/projects/9fda8c6d9ffa4955b2f19411ee340cb3/users/af5a109a6fb94042aeacbf141d7322ef/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.179760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:00 2022] PUT /identity/v3/projects/92f16adc2357459aa0afb68db89031dd/users/c2db6ada97da4ee89f3ce8de98f2653a/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:00.182154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef446-7a97-4e0d-896e-c031c4e1ac5c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.182600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef446-7a97-4e0d-896e-c031c4e1ac5c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.182892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef446-7a97-4e0d-896e-c031c4e1ac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.186880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93b00f-cc1e-46b1-a3c6-7d577131d554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fda8c6d9ffa4955b2f19411ee340cb3, user_id=af5a109a6fb94042aeacbf141d7322ef, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.190112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93b00f-cc1e-46b1-a3c6-7d577131d554 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.214856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 821/1609] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:00 2022] PUT /identity/v3/projects/9fda8c6d9ffa4955b2f19411ee340cb3/users/af5a109a6fb94042aeacbf141d7322ef/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:00.216340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa1f2a-e119-4a68-a684-aa363db03e51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.216661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa1f2a-e119-4a68-a684-aa363db03e51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.216966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa1f2a-e119-4a68-a684-aa363db03e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.225718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b11ef446-7a97-4e0d-896e-c031c4e1ac5c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:00.227110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b11ef446-7a97-4e0d-896e-c031c4e1ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.241298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1afa1f2a-e119-4a68-a684-aa363db03e51 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:00.242605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1afa1f2a-e119-4a68-a684-aa363db03e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.324476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 822/1610] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:00.326023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd672a-2558-4f14-afd9-24d163c26652 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.326319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd672a-2558-4f14-afd9-24d163c26652 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.326648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd672a-2558-4f14-afd9-24d163c26652 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.333072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 823/1611] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 05:21:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:00.334484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd672a-2558-4f14-afd9-24d163c26652 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.349367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1612] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:00.351238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2581c7-5cc3-4bd6-b606-a2ae5962bbf5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.351630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2581c7-5cc3-4bd6-b606-a2ae5962bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.351892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2581c7-5cc3-4bd6-b606-a2ae5962bbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.352004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd672a-2558-4f14-afd9-24d163c26652 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.358783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f709a03-6808-4b49-b840-5997b9675c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.359704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f709a03-6808-4b49-b840-5997b9675c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.360170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f709a03-6808-4b49-b840-5997b9675c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.360611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f709a03-6808-4b49-b840-5997b9675c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.361412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f709a03-6808-4b49-b840-5997b9675c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.364020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f709a03-6808-4b49-b840-5997b9675c02 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.379519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e2581c7-5cc3-4bd6-b606-a2ae5962bbf5 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:00.382008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e2581c7-5cc3-4bd6-b606-a2ae5962bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.392764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1613] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:00.398567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.398951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.399236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.415496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 824/1614] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:00.416941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.417326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.417668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.428656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:00.430595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.468210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:00.470019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.485422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:00.511299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1615] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:00.512382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.547283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:00.570670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 825/1616] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:00.579847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.594092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39400f85-a7a7-43da-bb3f-b7f7628f0d18 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.597912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.598546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.598929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.600567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.603639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1a17e-d401-4b8f-a4c2-ad7e5ea73162 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.608263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.610816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.611249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.611572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.612806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.614399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.628336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 826/1617] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:00.629514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.641391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.646788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58e3a4-ecc1-45e0-b178-31a22e9e74fb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.649621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.652025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.652602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.652886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.653405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.670637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.672935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.677429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:00.678362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.697454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 827/1619] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:00.698349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.699956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874525b-3ff8-434f-91d5-2975f05d6d92 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.701743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:00.708540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.709885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.710177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.710580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.711427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.713661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.734349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522636b-e438-46c5-a4d9-c64c05afd393 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.744437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.745087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.745400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.745652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.753589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1620] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:00 2022] POST /identity/v3/users => generated 293 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:00.754341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.766455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.768599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.774972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f316b-8c5c-469d-8759-6e0832c6b7b8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.783978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.785101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.785643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.786204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.795728 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 828/1621] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:00.796712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.817150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5b32a-2814-4f53-a865-af20dd34b27f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.827984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.828638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.828959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.829207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.829915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.831571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.860441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 829/1622] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:00.861240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.875198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.878257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.881787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae136-59bb-41ce-b51a-73b16002ee6f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:00.891091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1623] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:00.897313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:00.898034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:00.898327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:00.898616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] PATH_INFO: `/v3/projects/e86c96c4faf84a99976abb1fb1443c9b/users/b178ebc66a064d928c1badc47386dbb0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:00.916375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86c96c4faf84a99976abb1fb1443c9b, user_id=b178ebc66a064d928c1badc47386dbb0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:00.919866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:00.944227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 830/1624] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:00 2022] PUT /identity/v3/projects/e86c96c4faf84a99976abb1fb1443c9b/users/b178ebc66a064d928c1badc47386dbb0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:00.945689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:00.964470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818311b-abab-495a-8803-2fd54739d4da service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:01.106539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044752c-b13d-4a92-a68e-51ed84c5a7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:01.109191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044752c-b13d-4a92-a68e-51ed84c5a7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.109559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044752c-b13d-4a92-a68e-51ed84c5a7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.109874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044752c-b13d-4a92-a68e-51ed84c5a7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.110988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044752c-b13d-4a92-a68e-51ed84c5a7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:01.116623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044752c-b13d-4a92-a68e-51ed84c5a7a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:01.139877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1625] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:01.140567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:01.155030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f611512-86cd-470e-919e-479cb61ce7f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:01.169785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccac1de-ed65-4a81-bb20-f8f78dba09f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:01.170251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccac1de-ed65-4a81-bb20-f8f78dba09f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.170552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccac1de-ed65-4a81-bb20-f8f78dba09f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.170839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccac1de-ed65-4a81-bb20-f8f78dba09f8 admin admin] PATH_INFO: `/v3/projects/e86c96c4faf84a99976abb1fb1443c9b/users/b178ebc66a064d928c1badc47386dbb0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.176220 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccac1de-ed65-4a81-bb20-f8f78dba09f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86c96c4faf84a99976abb1fb1443c9b, user_id=b178ebc66a064d928c1badc47386dbb0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:01.179089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccac1de-ed65-4a81-bb20-f8f78dba09f8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:01.214090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 831/1626] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:01 2022] PUT /identity/v3/projects/e86c96c4faf84a99976abb1fb1443c9b/users/b178ebc66a064d928c1badc47386dbb0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:01.215971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.216516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.217031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.246063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:01.273967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] MFA Rules not processed for user `b178ebc66a064d928c1badc47386dbb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:01.276462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:01.350147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7e80a-24ab-48f7-9d8e-0c92616fc595 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.350468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7e80a-24ab-48f7-9d8e-0c92616fc595 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.351158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7e80a-24ab-48f7-9d8e-0c92616fc595 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.352173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 832/1627] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 05:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:21:01.362103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.362425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.362686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.395498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1628] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:01.409050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:01.413271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:01.414808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:01.509705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea8744-7554-4205-8fa9-9f18696f068c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:01.519084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:01.521879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.522334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.522898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.541123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:01.547858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:01.555674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 833/1629] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:21:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:01.570787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:01.571067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1630] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:01.583266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121d0af-19c0-456b-9ee8-31ddfc223fbb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:01.596167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fddcd-c9ab-4015-87c9-5dbc66239ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:01.596167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fddcd-c9ab-4015-87c9-5dbc66239ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:01.596167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fddcd-c9ab-4015-87c9-5dbc66239ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:01.596572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fddcd-c9ab-4015-87c9-5dbc66239ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:01.619525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fddcd-c9ab-4015-87c9-5dbc66239ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:01.621012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fddcd-c9ab-4015-87c9-5dbc66239ea3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:01.649261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 834/1631] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:02.461745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:02.620338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39c013-041b-4b17-ada3-0a9f72282cc2 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:02.638941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:02.639959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:02.640359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:02.640783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] PATH_INFO: `/v3/users/e20e6e74cb5d442ca07674bfa5627e41` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:02.645300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20e6e74cb5d442ca07674bfa5627e41)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:02.650861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:02.768596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:02.786457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20e6e74cb5d442ca07674bfa5627e41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:02.793461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20e6e74cb5d442ca07674bfa5627e41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:02.806958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20e6e74cb5d442ca07674bfa5627e41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:02.807691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20e6e74cb5d442ca07674bfa5627e41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:02.843646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20e6e74cb5d442ca07674bfa5627e41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:02.866341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1632] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:02 2022] DELETE /identity/v3/users/e20e6e74cb5d442ca07674bfa5627e41 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:02.868182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a011f3-69c1-454f-b1d4-59b021b09d91 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:02.868732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a011f3-69c1-454f-b1d4-59b021b09d91 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:02.868732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a011f3-69c1-454f-b1d4-59b021b09d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:02.908607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f9a011f3-69c1-454f-b1d4-59b021b09d91 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:02.910111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9a011f3-69c1-454f-b1d4-59b021b09d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:03.033547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:03.075144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 835/1633] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:03.090523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:03.092131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:03.222651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92324cfe-4cfb-4a72-a37a-153f036081d5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:03.223520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92324cfe-4cfb-4a72-a37a-153f036081d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:03.223520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92324cfe-4cfb-4a72-a37a-153f036081d5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:03.224966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 836/1634] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 05:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:21:03.235224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085d2e-c9f4-43cb-b7c7-bbfbb51fb7fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:03.235730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085d2e-c9f4-43cb-b7c7-bbfbb51fb7fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:03.235730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085d2e-c9f4-43cb-b7c7-bbfbb51fb7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:03.247468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec107140-677e-4cb4-add6-1b71b71ab38a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:03.256446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507f24-3fe9-4ca1-90e9-804d5793757a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:03.256818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507f24-3fe9-4ca1-90e9-804d5793757a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:03.257157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507f24-3fe9-4ca1-90e9-804d5793757a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:03.257380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507f24-3fe9-4ca1-90e9-804d5793757a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:03.302988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-71085d2e-c9f4-43cb-b7c7-bbfbb51fb7fa None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:03.310127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71085d2e-c9f4-43cb-b7c7-bbfbb51fb7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:03.313362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507f24-3fe9-4ca1-90e9-804d5793757a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:03.321552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507f24-3fe9-4ca1-90e9-804d5793757a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:03.352298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1635] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:03.445630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 837/1636] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 05:21:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:03.451663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6f82d-d61a-47a7-ad78-eb6ed9fee859 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:03.452398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6f82d-d61a-47a7-ad78-eb6ed9fee859 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:03.452694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6f82d-d61a-47a7-ad78-eb6ed9fee859 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:03.454145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1637] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 05:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:03.461003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:03.461266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:03.462281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:03.498392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:03.501524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:03.520703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 838/1638] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:21:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:03.532424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6f82d-d61a-47a7-ad78-eb6ed9fee859 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:03.537075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01c6f82d-d61a-47a7-ad78-eb6ed9fee859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:03.684746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6f82d-d61a-47a7-ad78-eb6ed9fee859 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:03.693033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804bb9-e3dc-4bae-be27-dddd8b10baad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:03.693855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804bb9-e3dc-4bae-be27-dddd8b10baad service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:03.694936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804bb9-e3dc-4bae-be27-dddd8b10baad service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:03.695267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804bb9-e3dc-4bae-be27-dddd8b10baad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:03.713562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804bb9-e3dc-4bae-be27-dddd8b10baad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:03.718629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804bb9-e3dc-4bae-be27-dddd8b10baad service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:03.774770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1639] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:04.562443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:04.567191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:04.634510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e32e0-b7d3-4a98-95fc-959e983244f6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:04.647208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:04.648099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:04.648390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:04.648676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] PATH_INFO: `/v3/projects/22cd7e817de842b58375aefa6b0f1ff4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:04.650206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22cd7e817de842b58375aefa6b0f1ff4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:04.652085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:04.885635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22cd7e817de842b58375aefa6b0f1ff4', 'request_id': 'req-2ba9bf35-1a52-4037-b1d0-f417e120d53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:04.888618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:04.889654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:04.890096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:04.955005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22cd7e817de842b58375aefa6b0f1ff4', 'request_id': 'req-2ba9bf35-1a52-4037-b1d0-f417e120d53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:04.975636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22cd7e817de842b58375aefa6b0f1ff4', 'request_id': 'req-2ba9bf35-1a52-4037-b1d0-f417e120d53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.011492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] MFA Rules not processed for user `47b7233b3a654d19a2182344fc5193a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:05.012981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.017162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] The token cache is being invalidate because project 22cd7e817de842b58375aefa6b0f1ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:05.017580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.018361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.020467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 839/1640] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:04 2022] DELETE /identity/v3/projects/22cd7e817de842b58375aefa6b0f1ff4 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:05.024172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.025816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.119231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:05.167041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1641] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:05.171839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.174733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.187464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:05.218135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9bf35-1a52-4037-b1d0-f417e120d53b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:05.224602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:05.225451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.225845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.226651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] PATH_INFO: `/v3/users/b178ebc66a064d928c1badc47386dbb0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.229856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b178ebc66a064d928c1badc47386dbb0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:05.232720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:21:05.232720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 29 05:21:05.232720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: Jul 29 05:21:05.234443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:05.345291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70372cd-19ec-4818-8f61-ec47cf5c2e21 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:05.354424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:05.355084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.355397 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.355725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.356368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:05.358342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:05.362921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b178ebc66a064d928c1badc47386dbb0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.377230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b178ebc66a064d928c1badc47386dbb0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.377961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b178ebc66a064d928c1badc47386dbb0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.378596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b178ebc66a064d928c1badc47386dbb0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.435837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1642] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:05 2022] POST /identity/v3/projects => generated 403 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:05.436911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.439837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.450251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b178ebc66a064d928c1badc47386dbb0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.508675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 840/1643] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:05 2022] DELETE /identity/v3/users/b178ebc66a064d928c1badc47386dbb0 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:05.513826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.518285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.604373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67f3d5-0ecf-4add-9208-28cbaeeb4138 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:05.671015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddfa93-70c1-4941-8e36-26604dbb639e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:05.684953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:05.685902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.686579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.688446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] PATH_INFO: `/v3/users/13be1d5917bc4a7ea41c83cce5c1e7f3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.689999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13be1d5917bc4a7ea41c83cce5c1e7f3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:05.696771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:05.723504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:05.724995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.725339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.725689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.726534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:05.728625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:05.777753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1644] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:05 2022] POST /identity/v3/users => generated 308 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:05.779219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.799488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13be1d5917bc4a7ea41c83cce5c1e7f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.807528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4344156-19c3-4016-a4f7-37a33bc93558 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:05.811864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13be1d5917bc4a7ea41c83cce5c1e7f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.814009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:05.814660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.814956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.815130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13be1d5917bc4a7ea41c83cce5c1e7f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.815232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.815870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13be1d5917bc4a7ea41c83cce5c1e7f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.815972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:05.817588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:05.827982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:05.837990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1645] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:05.840032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.841679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fdc1b14-6768-448a-9c54-f820ce1d0dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13be1d5917bc4a7ea41c83cce5c1e7f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:05.842415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.846232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:05.869006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74ec5-375c-476d-94a4-7487a9af34d4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:05.873134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 841/1646] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:05 2022] DELETE /identity/v3/users/13be1d5917bc4a7ea41c83cce5c1e7f3 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:05.877383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.877680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.878106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.883224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:05.884275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:05.884778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:05.885740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] PATH_INFO: `/v3/projects/95625b099c414ad98bfd57e39e7f1e11/users/f7d7dff7bfb24697ad3b74a4d07e0f2d/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:05.905990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:05.909318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:05.923369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95625b099c414ad98bfd57e39e7f1e11, user_id=f7d7dff7bfb24697ad3b74a4d07e0f2d, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:05.926508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:05.974527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1647] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:05 2022] PUT /identity/v3/projects/95625b099c414ad98bfd57e39e7f1e11/users/f7d7dff7bfb24697ad3b74a4d07e0f2d/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:05.975869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:05.996063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 842/1648] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:06.105983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:06.107764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:06.114475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d7529-7a44-4450-b6a1-dc37b7c43c9c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:06.121411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:06.122331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:06.122756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:06.123170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:06.124474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:06.126532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:06.143609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1649] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:06.152458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:06.169554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f62168-76e3-4657-86fb-ddb44e1338f1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:06.186230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:06.186230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:06.186565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:06.187009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] PATH_INFO: `/v3/projects/95625b099c414ad98bfd57e39e7f1e11/users/f7d7dff7bfb24697ad3b74a4d07e0f2d/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:06.190126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95625b099c414ad98bfd57e39e7f1e11, user_id=f7d7dff7bfb24697ad3b74a4d07e0f2d, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:06.195080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:06.217886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1650] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:06 2022] PUT /identity/v3/projects/95625b099c414ad98bfd57e39e7f1e11/users/f7d7dff7bfb24697ad3b74a4d07e0f2d/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:06.237481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5a38d-ae2e-42ae-9676-837fd6f01376 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:06.317056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a207-841e-49d0-b61c-9d71f821372a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:06.318615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a207-841e-49d0-b61c-9d71f821372a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:06.318985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a207-841e-49d0-b61c-9d71f821372a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:06.319518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a207-841e-49d0-b61c-9d71f821372a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:06.424864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4a207-841e-49d0-b61c-9d71f821372a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:06.428738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4a207-841e-49d0-b61c-9d71f821372a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:06.434295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:06.443782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 843/1651] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:08.001687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:08.004147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:08.031917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fbc1-e827-487a-9f21-a76379be5e65 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:08.042785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:08.044191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.044742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.045513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] PATH_INFO: `/v3/projects/e86c96c4faf84a99976abb1fb1443c9b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.046911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86c96c4faf84a99976abb1fb1443c9b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:08.048887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 05:21:08.048887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 29 05:21:08.048887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: Jul 29 05:21:08.050886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:08.223125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86c96c4faf84a99976abb1fb1443c9b', 'request_id': 'req-5b618f7a-a624-4daf-9a02-305fcdbbba0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.268748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86c96c4faf84a99976abb1fb1443c9b', 'request_id': 'req-5b618f7a-a624-4daf-9a02-305fcdbbba0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.290279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86c96c4faf84a99976abb1fb1443c9b', 'request_id': 'req-5b618f7a-a624-4daf-9a02-305fcdbbba0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.320302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] The token cache is being invalidate because project e86c96c4faf84a99976abb1fb1443c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:08.320746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.321631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5b618f7a-a624-4daf-9a02-305fcdbbba0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.328366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1652] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:08 2022] DELETE /identity/v3/projects/e86c96c4faf84a99976abb1fb1443c9b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:08.330119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.330592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.330673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.372694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:08.373981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:08.546603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 844/1653] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:08.556386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.556783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.557170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.609201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:08.611662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:08.648782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1654] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:08.651288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:08.671710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf19fb-9313-42dd-ba9d-cdc91aff15f9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:08.682525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:08.684722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:08.704411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:08.704726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465eca3-65ef-40e2-9c00-89b8bed4d505 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:08.705616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.709124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.709705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.710923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:08.715187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:08.715555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:08.716517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.716815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.717099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] PATH_INFO: `/v3/projects/908a005339464b1dacebb2fa00e0a14e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.720244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908a005339464b1dacebb2fa00e0a14e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:08.724610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:08.725754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:08.772703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 845/1655] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:21:08.776581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:08.802581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092e6b4-20c8-4d81-a5ae-3c166e05fbf9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:08.830173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:08.832036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:08.834488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.835025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.835916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.840061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:08.845942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:08.869404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908a005339464b1dacebb2fa00e0a14e', 'request_id': 'req-48bdee55-6e61-490f-b567-551ee8fa3280'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.884032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908a005339464b1dacebb2fa00e0a14e', 'request_id': 'req-48bdee55-6e61-490f-b567-551ee8fa3280'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.898345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908a005339464b1dacebb2fa00e0a14e', 'request_id': 'req-48bdee55-6e61-490f-b567-551ee8fa3280'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.921769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] The token cache is being invalidate because project 908a005339464b1dacebb2fa00e0a14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:08.922116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.922755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bdee55-6e61-490f-b567-551ee8fa3280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:08.926304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 846/1656] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:08 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:08.933242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1657] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:08 2022] DELETE /identity/v3/projects/908a005339464b1dacebb2fa00e0a14e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:08.933688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:08.935658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcbc27-6dc8-444d-a65f-19d6ce495c9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:08.935960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcbc27-6dc8-444d-a65f-19d6ce495c9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:08.936059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:08.936250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcbc27-6dc8-444d-a65f-19d6ce495c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:08.979787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69fcbc27-6dc8-444d-a65f-19d6ce495c9c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:08.983990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69fcbc27-6dc8-444d-a65f-19d6ce495c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:09.150136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2038df-cf3e-4b7a-b961-8e27c8cc6576 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.158074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1658] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:09.167039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.167980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.168514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.184103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.184103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.184103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.184103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.184526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.187573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.213763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:09.215498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:09.235341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1659] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:09.239950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.257265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 847/1660] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:08 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:09.258977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.272490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d79398-1828-46d9-bd12-8cf019b22a83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.278699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.279661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.280159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.280659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.281583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:09.282109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.284135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.290057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e55c48-d0f9-4e10-afd8-3bda88df5839 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.307188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1661] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:09.311539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.313659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.314338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.315457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.316745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.317633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.322418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.327056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742ba2-987c-4993-8c98-42139c127dd5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.339827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.340951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.341444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.341953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.344042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.347310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.361229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 848/1662] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:09.362477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.385638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f9a92-59ed-4a7b-9c85-ca97fb6acb55 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.402597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.405791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.409475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.409475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.409475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.424992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.448906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1663] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:09 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:09.456926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.471081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 849/1664] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:09.476903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.526899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7b3c3-25f4-482a-8f01-095c683c9e2d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.654502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:09.680078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22407b37-855e-4344-868c-4907ab8d3a97 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.700446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.701219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.701541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.701937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.703241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.704642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.714992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e32a9d-5ebb-4202-8ee5-9e873078f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:09.715760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32a9d-5ebb-4202-8ee5-9e873078f5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.716114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32a9d-5ebb-4202-8ee5-9e873078f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.716441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32a9d-5ebb-4202-8ee5-9e873078f5c8 admin admin] PATH_INFO: `/v3/projects/68834ebfe47b4d328ad93135d5fee32e/users/59acb39d791f400481580babf6ace511/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.750060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e32a9d-5ebb-4202-8ee5-9e873078f5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68834ebfe47b4d328ad93135d5fee32e, user_id=59acb39d791f400481580babf6ace511, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:09.758306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e32a9d-5ebb-4202-8ee5-9e873078f5c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:09.780484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1665] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:09 2022] POST /identity/v3/users => generated 284 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:09.792085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:09.795911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 850/1666] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:09 2022] PUT /identity/v3/projects/68834ebfe47b4d328ad93135d5fee32e/users/59acb39d791f400481580babf6ace511/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:09.811938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:09.812233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:09.812522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:09.820697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfed4-2282-445f-aefe-ddd38a9b831c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:09.828354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:09.869829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] MFA Rules not processed for user `59acb39d791f400481580babf6ace511`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:09.871100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:10.062612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 851/1667] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:10.068193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:10.080517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:10.081320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.082365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.088473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.090236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:10.094225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:10.112010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891b730-70c2-470d-9263-cbfd6ebf3402 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:10.129859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:10.133381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.133673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.134187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1668] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:10.134572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.135448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:10.137665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:10.138881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:10.179617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5fd94-00b4-4db5-88d2-3065bc0c7659 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:10.219775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:10.220474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.220763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.221039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.222629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:10.228197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:10.231194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 852/1669] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:10 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:10.235582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:10.277440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1670] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:10.278437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:10.313722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f56035-eb96-4049-bff7-af339257d7aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:10.500810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d0c6-00b9-484e-93fb-b287967c1922 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:10.534289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:10.559507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5d9f-9349-4aea-8cde-70e8f2235ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:10.560255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5d9f-9349-4aea-8cde-70e8f2235ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.560562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5d9f-9349-4aea-8cde-70e8f2235ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.561185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5d9f-9349-4aea-8cde-70e8f2235ec8 admin admin] PATH_INFO: `/v3/projects/e0884446279247f0a798be52172eafe6/users/58bbbfd4e806489b954ea90c25c55e34/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.563482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.564985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.566912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.568598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:10.579471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:10.627464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d5d9f-9349-4aea-8cde-70e8f2235ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0884446279247f0a798be52172eafe6, user_id=58bbbfd4e806489b954ea90c25c55e34, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:10.633544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d5d9f-9349-4aea-8cde-70e8f2235ec8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:10.661282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 853/1671] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:10 2022] POST /identity/v3/users => generated 289 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:10.662558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:10.670310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1672] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:10 2022] PUT /identity/v3/projects/e0884446279247f0a798be52172eafe6/users/58bbbfd4e806489b954ea90c25c55e34/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:10.682121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.682416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.682700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.721526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc573c9-b28a-434f-a3f2-44d529908eb9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:10.753702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:10.768794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] MFA Rules not processed for user `58bbbfd4e806489b954ea90c25c55e34`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:10.773246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:10.952533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:10.953190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:10.953585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:10.953871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:10.954593 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:10.960607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:10.996270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 854/1673] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:10.997254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:11.009685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1674] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:11.024958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:11.031139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:11.035817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1736608-d787-47ef-a495-fe44a9613220 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:11.061304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:11.062009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:11.062294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:11.062574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] PATH_INFO: `/v3/projects/02b96e07705c48f7b3f675a90ca7d72d/users/7427bde67333454f8a285acc626899dd/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:11.075046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:11.121802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b96e07705c48f7b3f675a90ca7d72d, user_id=7427bde67333454f8a285acc626899dd, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:11.127232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:11.179977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 855/1675] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:10 2022] PUT /identity/v3/projects/02b96e07705c48f7b3f675a90ca7d72d/users/7427bde67333454f8a285acc626899dd/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:11.180775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:11.259613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9c693-02c9-440d-86c8-92de48bbdef1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:11.331901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d6222-70cc-4676-85b2-7e749495820f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:11.353908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:11.354570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:11.354856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:11.355134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:11.355869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:11.368391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:11.402671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 856/1676] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:11.405189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:11.444644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfda41-ac11-4a1c-8657-8dbefdaea78d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:11.449634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:11.450305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:11.450594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:11.450881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:11.456830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237843da-74ef-4a2b-8502-e3455faed69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:11.457535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237843da-74ef-4a2b-8502-e3455faed69d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:11.457822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237843da-74ef-4a2b-8502-e3455faed69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:11.458690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237843da-74ef-4a2b-8502-e3455faed69d admin admin] PATH_INFO: `/v3/projects/02b96e07705c48f7b3f675a90ca7d72d/users/7427bde67333454f8a285acc626899dd/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:11.462496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237843da-74ef-4a2b-8502-e3455faed69d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b96e07705c48f7b3f675a90ca7d72d, user_id=7427bde67333454f8a285acc626899dd, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:11.470837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237843da-74ef-4a2b-8502-e3455faed69d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:11.521556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 857/1677] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:11 2022] PUT /identity/v3/projects/02b96e07705c48f7b3f675a90ca7d72d/users/7427bde67333454f8a285acc626899dd/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:11.529120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce01a8-d159-4a8c-880e-7b91a4fb94ec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:11.529450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce01a8-d159-4a8c-880e-7b91a4fb94ec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:11.529738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce01a8-d159-4a8c-880e-7b91a4fb94ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:11.606365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-30ce01a8-d159-4a8c-880e-7b91a4fb94ec None None] MFA Rules not processed for user `7427bde67333454f8a285acc626899dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:11.607905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30ce01a8-d159-4a8c-880e-7b91a4fb94ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:11.683326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:11.685079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:11.703698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1678] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:11.759309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:11.802369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 858/1679] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:11.821113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:11.822594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:11.841840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cc7c0-071d-4756-8e7c-4095d5b391a8 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:11.951750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:11.952540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:11.952815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:11.953097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:11.982118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:11.983818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:12.010143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1680] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:12.099403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630814bd-01b9-4c50-81c7-10330b673b46 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:12.099707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630814bd-01b9-4c50-81c7-10330b673b46 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:12.100002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630814bd-01b9-4c50-81c7-10330b673b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:12.163270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-630814bd-01b9-4c50-81c7-10330b673b46 None None] MFA Rules not processed for user `f7d7dff7bfb24697ad3b74a4d07e0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:12.164953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-630814bd-01b9-4c50-81c7-10330b673b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:12.214654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:12.225801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4a2cd-8e38-45e2-a849-03ca3693d1ed service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:12.238355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:12.238997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:12.242909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:12.243213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:12.291078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:12.383658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 859/1681] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:12 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:12.394280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:12.395935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:12.416546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1682] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:13.418890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc906d43-360e-4c09-8aab-b809f6f5fd3b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:13.419383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc906d43-360e-4c09-8aab-b809f6f5fd3b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:13.419588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc906d43-360e-4c09-8aab-b809f6f5fd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:13.476177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dc906d43-360e-4c09-8aab-b809f6f5fd3b None None] MFA Rules not processed for user `3c60b3a983f14f50b1b762a3954a1059`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:13.477637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc906d43-360e-4c09-8aab-b809f6f5fd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:13.715255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 860/1683] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:13.745078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:13.749374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:13.963829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1601ff-7f7f-48a7-bf14-36ca736f4bc8 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:13.985650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b4348-b103-449b-aab9-18fcbc0ac48c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:13.986319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b4348-b103-449b-aab9-18fcbc0ac48c service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:13.986944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b4348-b103-449b-aab9-18fcbc0ac48c service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:13.987246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b4348-b103-449b-aab9-18fcbc0ac48c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:14.009456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b4348-b103-449b-aab9-18fcbc0ac48c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:14.016889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b4348-b103-449b-aab9-18fcbc0ac48c service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:14.055195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1684] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:15.076323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f472d9-ff35-41e8-84fd-f59470525576 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:15.077890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f472d9-ff35-41e8-84fd-f59470525576 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:15.078363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f472d9-ff35-41e8-84fd-f59470525576 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:15.080263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 861/1685] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:21:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:15.175469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f269b-d815-407c-8f99-0b8ffef52e99 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:15.175959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f269b-d815-407c-8f99-0b8ffef52e99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:15.176178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f269b-d815-407c-8f99-0b8ffef52e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:15.225520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca9f269b-d815-407c-8f99-0b8ffef52e99 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:15.230539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca9f269b-d815-407c-8f99-0b8ffef52e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:15.427946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1686] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:21:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:15.699418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd4114-3165-410f-b440-cd647d69b480 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:15.700011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd4114-3165-410f-b440-cd647d69b480 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:15.700380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd4114-3165-410f-b440-cd647d69b480 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:15.702291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 862/1687] 10.222.0.49 () {54 vars in 961 bytes} [Fri Jul 29 05:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:15.715769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:15.716390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:15.716891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:15.764472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:15.765755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:15.802822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1688] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Jul 29 05:21:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:15.825681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd4114-3165-410f-b440-cd647d69b480 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:15.871742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd4114-3165-410f-b440-cd647d69b480 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:15.890688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096bba4-aa72-434c-b10a-af477bb8cc3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:15.891408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096bba4-aa72-434c-b10a-af477bb8cc3d service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:15.891914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096bba4-aa72-434c-b10a-af477bb8cc3d service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:15.892202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096bba4-aa72-434c-b10a-af477bb8cc3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:15.917607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096bba4-aa72-434c-b10a-af477bb8cc3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:15.919952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096bba4-aa72-434c-b10a-af477bb8cc3d service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:15.961450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 863/1689] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:16.837417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:16.839451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:16.976218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:17.070644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778c8b7-d773-4328-b19d-a26d04ea1713 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:17.093927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:17.094942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:17.095486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:17.095972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] PATH_INFO: `/v3/users/47b7233b3a654d19a2182344fc5193a7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:17.097742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b7233b3a654d19a2182344fc5193a7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:17.100561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:17.189991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:17.190912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:17.191486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:17.223782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b7233b3a654d19a2182344fc5193a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:17.229729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b7233b3a654d19a2182344fc5193a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:17.248427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b7233b3a654d19a2182344fc5193a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:17.257043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b7233b3a654d19a2182344fc5193a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:17.275640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] MFA Rules not processed for user `45ae88234e984149be8539757efa8733`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:17.282421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:17.290181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-46d6ae8b-9b3a-4ea8-8d3d-9ad68ca72592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b7233b3a654d19a2182344fc5193a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:17.325891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1690] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:16 2022] DELETE /identity/v3/users/47b7233b3a654d19a2182344fc5193a7 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:17.422482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:17.486851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 864/1691] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:17.495669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:17.496085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:17.496438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:17.534343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] MFA Rules not processed for user `45ae88234e984149be8539757efa8733`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:17.536775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:17.727178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1692] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:17.737717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:17.740771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:17.789281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad98be-3f45-4794-aab0-70d73274314d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:18.011801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:18.012614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:18.016975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:18.019374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:18.046566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:18.053758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:18.111428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 865/1693] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:18.842075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:18.846405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:19.033464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:19.060009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d9e9d-88f7-4994-9619-7979e813dc94 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:19.077075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:19.078043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:19.078838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:19.079628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] PATH_INFO: `/v3/projects/91858dac25b4484e95e7354ade64775d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:19.082101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91858dac25b4484e95e7354ade64775d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:19.085609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:19.220832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91858dac25b4484e95e7354ade64775d', 'request_id': 'req-202b24f9-0644-4b19-90c0-7a637780e680'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.241542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91858dac25b4484e95e7354ade64775d', 'request_id': 'req-202b24f9-0644-4b19-90c0-7a637780e680'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.258525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91858dac25b4484e95e7354ade64775d', 'request_id': 'req-202b24f9-0644-4b19-90c0-7a637780e680'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.284493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] The token cache is being invalidate because project 91858dac25b4484e95e7354ade64775d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:19.284950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.286181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-202b24f9-0644-4b19-90c0-7a637780e680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.292193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1694] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:18 2022] DELETE /identity/v3/projects/91858dac25b4484e95e7354ade64775d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:19.296783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:19.323144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eed18-d116-42a0-a585-29c0974fc971 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:19.475263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:19.476000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:19.476340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:19.476651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] PATH_INFO: `/v3/users/f7d7dff7bfb24697ad3b74a4d07e0f2d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:19.486526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d7dff7bfb24697ad3b74a4d07e0f2d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:19.490804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:19.535513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca32f88-dde0-4aa0-979d-12d88a15d725 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:19.535921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca32f88-dde0-4aa0-979d-12d88a15d725 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:19.536208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca32f88-dde0-4aa0-979d-12d88a15d725 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:19.538114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1695] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 05:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:19.547288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:19.547288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:19.547645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:19.656944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d7dff7bfb24697ad3b74a4d07e0f2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.669911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:19.673871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d7dff7bfb24697ad3b74a4d07e0f2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.675030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d7dff7bfb24697ad3b74a4d07e0f2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.675855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d7dff7bfb24697ad3b74a4d07e0f2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.683748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:19.801346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:19.805458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89e675c0-df77-42a6-a012-5fd999d164c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d7dff7bfb24697ad3b74a4d07e0f2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:19.868099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 866/1696] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:19 2022] DELETE /identity/v3/users/f7d7dff7bfb24697ad3b74a4d07e0f2d => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:19.882866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed88b7f-70cb-4556-81a9-4e627562d6d9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:19.885000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed88b7f-70cb-4556-81a9-4e627562d6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:19.885300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed88b7f-70cb-4556-81a9-4e627562d6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:19.945498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1697] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:21:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:20.010010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5ed88b7f-70cb-4556-81a9-4e627562d6d9 None None] MFA Rules not processed for user `720be309cbe14940873d774b06e91bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:20.011204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ed88b7f-70cb-4556-81a9-4e627562d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:20.051452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:20.053000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:20.088812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7a2d-24a2-48c1-8634-39de46b2428f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:20.199418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:20.199641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:20.200268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:20.200622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:20.232619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:20.235516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 867/1698] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:20.253249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:20.255408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:20.258072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:20.258419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:20.276156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1699] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:20.297687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] MFA Rules not processed for user `720be309cbe14940873d774b06e91bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:20.299569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:20.329110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 868/1700] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:20.347053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:20.349774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:20.433914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:20.613109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027671fd-7553-4b9a-9534-7dc9281c0040 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:20.635863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1814b1-9734-4a07-bd8c-741879c7e270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:20.636555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1814b1-9734-4a07-bd8c-741879c7e270 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:20.636841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1814b1-9734-4a07-bd8c-741879c7e270 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:20.639802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1814b1-9734-4a07-bd8c-741879c7e270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:20.669047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1814b1-9734-4a07-bd8c-741879c7e270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:20.673942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1814b1-9734-4a07-bd8c-741879c7e270 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:20.709564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1701] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:21.304310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:21.305773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:21.500200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d304888-ba47-4b68-b355-b5289b8da774 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:21.516431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:21.517310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:21.518048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:21.518445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] PATH_INFO: `/v3/projects/95625b099c414ad98bfd57e39e7f1e11` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:21.520847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95625b099c414ad98bfd57e39e7f1e11)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:21.525442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:21.741625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95625b099c414ad98bfd57e39e7f1e11', 'request_id': 'req-c96dea63-d232-476b-9793-7e4fe828b4b5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:21.769056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95625b099c414ad98bfd57e39e7f1e11', 'request_id': 'req-c96dea63-d232-476b-9793-7e4fe828b4b5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:21.792696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4667d8-aae6-4b93-955e-7bd4f15c0977 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:21.794138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4667d8-aae6-4b93-955e-7bd4f15c0977 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:21.795033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4667d8-aae6-4b93-955e-7bd4f15c0977 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:21.798845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1702] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 05:21:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:21:21.807175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87050f-adf6-4955-b7fe-cd3fd3ab141b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:21.807502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87050f-adf6-4955-b7fe-cd3fd3ab141b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:21.807783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87050f-adf6-4955-b7fe-cd3fd3ab141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:21.816005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95625b099c414ad98bfd57e39e7f1e11', 'request_id': 'req-c96dea63-d232-476b-9793-7e4fe828b4b5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:21.872207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:21.895672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] The token cache is being invalidate because project 95625b099c414ad98bfd57e39e7f1e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:21.895672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:21.896493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c96dea63-d232-476b-9793-7e4fe828b4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:21.904454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 869/1703] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:21 2022] DELETE /identity/v3/projects/95625b099c414ad98bfd57e39e7f1e11 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:21.910816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71556240-59cb-4000-849e-e76fef171176 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:21.912344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b87050f-adf6-4955-b7fe-cd3fd3ab141b None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:21.922784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71556240-59cb-4000-849e-e76fef171176 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:21.923254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71556240-59cb-4000-849e-e76fef171176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:21.924855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b87050f-adf6-4955-b7fe-cd3fd3ab141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:21.990568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-71556240-59cb-4000-849e-e76fef171176 None None] MFA Rules not processed for user `45ae88234e984149be8539757efa8733`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:21.992021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71556240-59cb-4000-849e-e76fef171176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:22.173285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 870/1704] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:22.174371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71556240-59cb-4000-849e-e76fef171176 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:22.178769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71556240-59cb-4000-849e-e76fef171176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:22.192995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1705] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:21:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:22.194734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76412c8-6ee4-4c18-9718-99337233ae88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:22.198782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76412c8-6ee4-4c18-9718-99337233ae88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:22.200012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76412c8-6ee4-4c18-9718-99337233ae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:22.281783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d76412c8-6ee4-4c18-9718-99337233ae88 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:22.286953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d76412c8-6ee4-4c18-9718-99337233ae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:22.450645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71556240-59cb-4000-849e-e76fef171176 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:22.476165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:22.477137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:22.477654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:22.478149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:22.480929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:22.515094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1706] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:22.526010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:22.526722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:22.527575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:22.550300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:22.552924 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:22.589161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:22.590780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:22.612348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 871/1707] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:22.632464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1708] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:22.637227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:22.680565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e98f8-69c9-4298-b546-e4f368047c73 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:22.720164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:22.720421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:22.721881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:22.724461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:22.729883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:22.732317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:22.734309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:22.735173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:22.740156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:22.751959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41166080-6a3d-4c86-bd12-96ab85b29a96 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:22.766335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:22.767536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:22.767536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:22.780654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:22.782134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 872/1709] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:22.787817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:22.802027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:22.813226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:22.832522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1710] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:22.836901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530672b0-7cdc-4e94-b62b-262506d977df admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:22.858782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:22.859156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:22.859639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:22.860319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:22.862587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:22.865848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:22.927603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 873/1711] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:22 2022] POST /identity/v3/projects => generated 355 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:22.929326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:22.951176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b66080-6396-4684-b0ff-57eb6082bd26 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:23.078909 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:23.159868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:23.160717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:23.161217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:23.161519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:23.162226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:23.164484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:23.213718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1712] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:22 2022] POST /identity/v3/users => generated 278 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:23.217657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:23.234494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4633bb2-0736-442e-87e3-a83fe1850a17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:23.250214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:23.250867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:23.251151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:23.251448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:23.252161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:23.253820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:23.292502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 874/1713] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:23.296708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:23.312558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bf065-0b85-4256-8ba3-277330d4ac51 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:23.337055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380cac4-50d8-47a0-a112-917428072d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:23.337055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380cac4-50d8-47a0-a112-917428072d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:23.337055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380cac4-50d8-47a0-a112-917428072d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:23.337055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380cac4-50d8-47a0-a112-917428072d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:23.338157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380cac4-50d8-47a0-a112-917428072d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:23.338157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380cac4-50d8-47a0-a112-917428072d49 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:23.378509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1714] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:23.384561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:23.408820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad966-3a6b-4710-8a44-dae117565eb1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:23.437288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d25e9-d2be-46f0-b465-e103434a638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:23.439072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d25e9-d2be-46f0-b465-e103434a638f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:23.440358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d25e9-d2be-46f0-b465-e103434a638f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:23.442583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d25e9-d2be-46f0-b465-e103434a638f admin admin] PATH_INFO: `/v3/projects/a67a3b58e8c04374abdcd8fec4e0b411/users/ba8749223fd44b29906737989dc22438/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:23.525135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d25e9-d2be-46f0-b465-e103434a638f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a67a3b58e8c04374abdcd8fec4e0b411, user_id=ba8749223fd44b29906737989dc22438, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:23.557504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d25e9-d2be-46f0-b465-e103434a638f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:23.624448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 875/1715] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:23 2022] PUT /identity/v3/projects/a67a3b58e8c04374abdcd8fec4e0b411/users/ba8749223fd44b29906737989dc22438/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:23.626603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:23.627271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:23.627764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:23.695468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] MFA Rules not processed for user `ba8749223fd44b29906737989dc22438`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:23.696746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:23.845422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:23.845701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:23.846050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:23.910998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] MFA Rules not processed for user `3c60b3a983f14f50b1b762a3954a1059`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:23.916190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:23.919926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1716] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:23.925820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:23.972145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:24.072632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 876/1717] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:24.089464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c1e2f-7bc6-4b6f-ae1c-89e2190fd18c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:24.101753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:24.102712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:24.103198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:24.103774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:24.104694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:24.110805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:24.188047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1718] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:23 2022] POST /identity/v3/projects => generated 355 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:24.194162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:24.219988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f5b32-301f-40af-bb06-52842dc3be42 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:24.443802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:24.444906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:24.445292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:24.445681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:24.446500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:24.452395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:24.537802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 877/1719] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:24 2022] POST /identity/v3/users => generated 284 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:24.543653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:24.566373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790f51-fee5-4df5-9838-3f6169612efe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:24.594460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:24.594952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:24.595380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:24.595767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:24.596592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:24.610612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:24.657001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1720] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:24.658365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:24.701327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03989f15-a13a-4979-9cc5-df57f0321115 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:24.706061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:24.708622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:24.734247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:24.736591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:24.738361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:24.740539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] PATH_INFO: `/v3/projects/85eb103fb0934fffa765d31b0c8808e4/users/f8847b3ba3e6409d866165e6e9fd421b/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:24.752004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07102-da5b-4ae5-abd6-6fd35c9f9d38 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:24.790383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:24.795296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:24.798556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:24.798789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] PATH_INFO: `/v3/users/58bbbfd4e806489b954ea90c25c55e34` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:24.819230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58bbbfd4e806489b954ea90c25c55e34)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:24.823320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:24.834283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eb103fb0934fffa765d31b0c8808e4, user_id=f8847b3ba3e6409d866165e6e9fd421b, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:24.847145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:24.901759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 878/1721] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:24 2022] PUT /identity/v3/projects/85eb103fb0934fffa765d31b0c8808e4/users/f8847b3ba3e6409d866165e6e9fd421b/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:24.902867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:24.908611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:25.042515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58bbbfd4e806489b954ea90c25c55e34'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:25.044936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58bbbfd4e806489b954ea90c25c55e34'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:25.091167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58bbbfd4e806489b954ea90c25c55e34'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:25.119258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382db60-ae36-4944-8c80-432316ce462c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:25.123640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58bbbfd4e806489b954ea90c25c55e34'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:25.140881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:25.141798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:25.142378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:25.142808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:25.143952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:25.147071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:25.167208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 879/1722] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:25.174483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:25.180864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:25.237858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eee0d2-f3c9-4df6-8743-085a18871546 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:25.263853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ade5f-a253-4acd-94c5-a62f5813a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:25.264622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ade5f-a253-4acd-94c5-a62f5813a7be admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:25.264916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ade5f-a253-4acd-94c5-a62f5813a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:25.265221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ade5f-a253-4acd-94c5-a62f5813a7be admin admin] PATH_INFO: `/v3/projects/85eb103fb0934fffa765d31b0c8808e4/users/f8847b3ba3e6409d866165e6e9fd421b/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:25.270644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ade5f-a253-4acd-94c5-a62f5813a7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eb103fb0934fffa765d31b0c8808e4, user_id=f8847b3ba3e6409d866165e6e9fd421b, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:25.281993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ade5f-a253-4acd-94c5-a62f5813a7be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:25.306066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8da501ad-3fb5-456b-9073-b5f9142f24f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58bbbfd4e806489b954ea90c25c55e34'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:25.311708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 880/1723] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:25 2022] PUT /identity/v3/projects/85eb103fb0934fffa765d31b0c8808e4/users/f8847b3ba3e6409d866165e6e9fd421b/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:25.338505 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:25.339218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:25.341278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:25.391391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1724] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:24 2022] DELETE /identity/v3/users/58bbbfd4e806489b954ea90c25c55e34 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:25.412698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:25.412996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:25.413283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:25.433171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] MFA Rules not processed for user `f8847b3ba3e6409d866165e6e9fd421b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:25.434411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:25.442608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:25.499542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:25.501216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:25.603590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 881/1725] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:25 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:25.636930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:25.647019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:25.792582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1726] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:25.936265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1ed16-d7a9-4025-9b63-dd89e53ce728 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:25.957765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:25.960071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:25.961662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:25.962161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:25.967076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:25.968587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:26.017708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:26.029731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:26.104725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 882/1727] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:25 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:26.149962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e45c4-b2dc-4537-8b38-cd6898b7653e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:26.163314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99959dc-b1bd-49f4-bce1-eafcfef8dab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:26.164269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99959dc-b1bd-49f4-bce1-eafcfef8dab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:26.164554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99959dc-b1bd-49f4-bce1-eafcfef8dab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:26.164912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99959dc-b1bd-49f4-bce1-eafcfef8dab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:26.191205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99959dc-b1bd-49f4-bce1-eafcfef8dab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:26.193743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99959dc-b1bd-49f4-bce1-eafcfef8dab8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:26.219011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1728] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:26.415602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:26.448963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:26.449445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:26.449863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:26.459703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a1be-4ac6-4723-af5f-33144f7563c3 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:26.481972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:26.483493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:26.485250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:26.486481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:26.509074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:26.536430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] MFA Rules not processed for user `af5a109a6fb94042aeacbf141d7322ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:26.537752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:26.664253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:26.777004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1729] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:26.777908 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:26.784286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:26.842394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5308dd-b515-423d-b3e8-8440bdbcba12 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:26.858873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:26.859593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:26.859891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:26.860177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] PATH_INFO: `/v3/users/59acb39d791f400481580babf6ace511` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:26.864405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59acb39d791f400481580babf6ace511)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:26.866387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:26.939009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:26.947021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:26.974754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 883/1730] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:26 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:26.979667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:26.982074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:27.092171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59acb39d791f400481580babf6ace511'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:27.097375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59acb39d791f400481580babf6ace511'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:27.139794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:27.152250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59acb39d791f400481580babf6ace511'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:27.158889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59acb39d791f400481580babf6ace511'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:27.252429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8822d30-5ce8-46d7-bb69-9edb27355f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59acb39d791f400481580babf6ace511'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:27.261372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba1bb4-90d3-4ef3-9f7f-4f57a2eeb301 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:27.295881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:27.296763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:27.297041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:27.297334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:27.305931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:27.338736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1731] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:26 2022] DELETE /identity/v3/users/59acb39d791f400481580babf6ace511 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:27.348099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:27.348723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:27.349380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca4cf0-13ac-4475-bc51-cba95ff2312c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:27.349547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:27.352593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:27.383572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 884/1732] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:27.537080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:27.542975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:27.741253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc9c7-e814-4a50-ba9a-b2e556d98680 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:27.741550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc9c7-e814-4a50-ba9a-b2e556d98680 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:27.741850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc9c7-e814-4a50-ba9a-b2e556d98680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:27.817453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1733] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:27.843053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-29ebc9c7-e814-4a50-ba9a-b2e556d98680 None None] MFA Rules not processed for user `c2db6ada97da4ee89f3ce8de98f2653a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:27.844362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-29ebc9c7-e814-4a50-ba9a-b2e556d98680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:27.916507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:27.919106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:28.159542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:28.179099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 885/1734] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:28.180649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:28.183265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:28.183643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:28.236820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778d40f-c334-4105-9a80-8c796733c681 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:28.257393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:28.258316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:28.258774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:28.259205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:28.285015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:28.290128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:28.315828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1735] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:28.345036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:28.345751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:28.532731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 886/1736] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:21:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:28.904866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:28.908861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:29.337007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:29.339775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:29.438825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f243e-1939-44bb-bf1c-736daf355b4b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:29.468558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:29.469604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:29.469755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:29.470188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:29.500574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:29.502309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:29.511766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:29.564349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1737] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:21:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:21:29.750413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca2ef7-c205-40f3-99f8-626cca6df02a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:29.772169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:29.772998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:29.773501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:29.773926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:29.815203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:29.818018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:29.862666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 887/1738] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:30.095559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:30.098634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:30.135462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec7f05-1474-44a7-b2bb-401fab45fd3c service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:30.183869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:30.189448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:30.200904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:30.201422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] PATH_INFO: `/v3/projects/e0884446279247f0a798be52172eafe6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:30.203988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0884446279247f0a798be52172eafe6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:30.206158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:30.436489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0884446279247f0a798be52172eafe6', 'request_id': 'req-16ab1797-2552-459b-a287-ec0f12576a86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:30.465605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0884446279247f0a798be52172eafe6', 'request_id': 'req-16ab1797-2552-459b-a287-ec0f12576a86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:30.481514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0884446279247f0a798be52172eafe6', 'request_id': 'req-16ab1797-2552-459b-a287-ec0f12576a86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:30.500351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:30.504349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:30.512655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] The token cache is being invalidate because project e0884446279247f0a798be52172eafe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:30.513259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:30.515878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16ab1797-2552-459b-a287-ec0f12576a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:30.520041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1739] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:30 2022] DELETE /identity/v3/projects/e0884446279247f0a798be52172eafe6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:30.530798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159fd4-082c-4bbd-b37e-d1414467cde2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:30.530798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159fd4-082c-4bbd-b37e-d1414467cde2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:30.531042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159fd4-082c-4bbd-b37e-d1414467cde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:30.547937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed7ba0-d1ee-4e88-8087-720b33f2fd25 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:30.641872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b9159fd4-082c-4bbd-b37e-d1414467cde2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:30.643823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9159fd4-082c-4bbd-b37e-d1414467cde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:30.770678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:30.833451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:30.834168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:30.834456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:30.834739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] PATH_INFO: `/v3/projects/68834ebfe47b4d328ad93135d5fee32e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:30.842706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=68834ebfe47b4d328ad93135d5fee32e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:30.845460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:30.918830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1740] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:30.933578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:30.933875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:30.934161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:31.020111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:31.021536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:31.097443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68834ebfe47b4d328ad93135d5fee32e', 'request_id': 'req-e6251cc0-39e2-404f-b3d4-672e29176d3b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.120537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68834ebfe47b4d328ad93135d5fee32e', 'request_id': 'req-e6251cc0-39e2-404f-b3d4-672e29176d3b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.145411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68834ebfe47b4d328ad93135d5fee32e', 'request_id': 'req-e6251cc0-39e2-404f-b3d4-672e29176d3b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.205126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] The token cache is being invalidate because project 68834ebfe47b4d328ad93135d5fee32e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:31.205500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.209030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.216020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 888/1741] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:30 2022] DELETE /identity/v3/projects/68834ebfe47b4d328ad93135d5fee32e => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:31.220291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:31.224582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:31.239227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1742] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:31.243077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:31.269990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6251cc0-39e2-404f-b3d4-672e29176d3b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:31.277354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb6a5a-80c6-45d8-a18b-7d21dcf1e306 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:31.296153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:31.302224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:31.302801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:31.303236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:31.306176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:31.308732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:31.309348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:31.309465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:31.309687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:31.309968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] PATH_INFO: `/v3/users/7427bde67333454f8a285acc626899dd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:31.315725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7427bde67333454f8a285acc626899dd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:31.316788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:31.318628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:31.350659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1743] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:31.355671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:31.382471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce95e9-d4dd-464f-9b65-f80ea422e25e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:31.394003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:31.396123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:31.397927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:31.398444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:31.399391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:31.406336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:31.478577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1744] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:31 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:31.487507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:31.542277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:31.568654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7427bde67333454f8a285acc626899dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.607513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7427bde67333454f8a285acc626899dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.608799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7427bde67333454f8a285acc626899dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.611450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7427bde67333454f8a285acc626899dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.683904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857f8b-cc7e-431b-9a98-1e5706abd46c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:31.685369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7427bde67333454f8a285acc626899dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:31.706932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:31.708025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:31.712127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:31.716418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:31.718715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:31.720920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:31.756237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 889/1745] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:31 2022] DELETE /identity/v3/users/7427bde67333454f8a285acc626899dd => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:31.803656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1746] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:31 2022] POST /identity/v3/users => generated 291 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:31.809140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:31.818121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:32.181904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750f828-1d3b-4d27-9092-b62ff15dfd81 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:32.225263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:32.228641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:32.229034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:32.232963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:32.233990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:32.240383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:32.285854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 890/1747] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:32.290382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:32.305931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b94059-5964-49d3-a3da-288f91af5bf5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:32.333066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc7893-3cc8-4201-a292-5cc838f32437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:32.335466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7893-3cc8-4201-a292-5cc838f32437 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:32.336748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7893-3cc8-4201-a292-5cc838f32437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:32.337877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7893-3cc8-4201-a292-5cc838f32437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:32.339695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc7893-3cc8-4201-a292-5cc838f32437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:32.343944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc7893-3cc8-4201-a292-5cc838f32437 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:32.385442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1748] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:32.389410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:32.406232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d07b1-9658-4bcf-b95d-b863c952c191 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:32.426702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:32.427574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:32.427870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:32.428212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] PATH_INFO: `/v3/projects/c5d25eeaa5e346458230dfc09fa303ed/users/96ff5b6366634481b30dabd37556fd48/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:32.492475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d25eeaa5e346458230dfc09fa303ed, user_id=96ff5b6366634481b30dabd37556fd48, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:32.495261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:32.555476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 891/1749] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:32 2022] PUT /identity/v3/projects/c5d25eeaa5e346458230dfc09fa303ed/users/96ff5b6366634481b30dabd37556fd48/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:32.567150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:32.567821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:32.567821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:32.749346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] MFA Rules not processed for user `96ff5b6366634481b30dabd37556fd48`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:32.750598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:32.767812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:33.060015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1750] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:33.060977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:33.361797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8c4c3-49a8-450e-ae15-b8ce890074c9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:33.374871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:33.376988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:33.377455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:33.377900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:33.379575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:33.386441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:33.497313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:33.497998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 892/1751] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:33 2022] POST /identity/v3/projects => generated 383 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:33.560201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a9577-d983-45b1-8804-4c7e53cd6c60 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:33.741284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f566-f752-4f37-9ce9-63b2d5bd31a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:33.741975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f566-f752-4f37-9ce9-63b2d5bd31a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:33.742268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f566-f752-4f37-9ce9-63b2d5bd31a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:33.742497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f566-f752-4f37-9ce9-63b2d5bd31a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:33.743202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3f566-f752-4f37-9ce9-63b2d5bd31a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:33.745364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3f566-f752-4f37-9ce9-63b2d5bd31a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:33.801384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1752] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:33 2022] POST /identity/v3/users => generated 298 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:33.805851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:33.830319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e4242-7fef-4eaa-a390-d6764a9caf84 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:33.844398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:33.848942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:33.850164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:33.850745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:33.851823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:33.853915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:33.854899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:33.855203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:33.855527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:33.904183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 893/1753] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:33.909440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:33.920888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:33.922172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:33.930276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:33.939037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3440-14a4-49a1-a14f-7c32d09a4cc0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:33.958607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:33.959546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:33.960179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:33.961089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] PATH_INFO: `/v3/projects/6f4c8861e1094021bf58874d58a5a132/users/fa0e6b920dd24011ac80bdffc76f500c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:34.062256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4c8861e1094021bf58874d58a5a132, user_id=fa0e6b920dd24011ac80bdffc76f500c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:34.072220 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:34.103073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 894/1754] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:33 2022] PUT /identity/v3/projects/6f4c8861e1094021bf58874d58a5a132/users/fa0e6b920dd24011ac80bdffc76f500c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:34.107935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:34.115029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1755] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Jul 29 05:21:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:34.343507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350eb18-f0a4-4e19-b9b5-3c8dd5ab1b8b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:34.362993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc23150-8616-4298-9591-2b6ddc89084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:34.370509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23150-8616-4298-9591-2b6ddc89084b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:34.370509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23150-8616-4298-9591-2b6ddc89084b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:34.370509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23150-8616-4298-9591-2b6ddc89084b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:34.371860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc23150-8616-4298-9591-2b6ddc89084b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:34.375474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc23150-8616-4298-9591-2b6ddc89084b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:34.407329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 895/1756] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:34.410102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:34.432553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93a9c0-c383-4ea7-b65b-78a9cca31b14 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:34.440709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:34.443262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:34.443583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:34.444033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] PATH_INFO: `/v3/projects/6f4c8861e1094021bf58874d58a5a132/users/fa0e6b920dd24011ac80bdffc76f500c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:34.449637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4c8861e1094021bf58874d58a5a132, user_id=fa0e6b920dd24011ac80bdffc76f500c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:34.453860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:34.506839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1757] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:34 2022] PUT /identity/v3/projects/6f4c8861e1094021bf58874d58a5a132/users/fa0e6b920dd24011ac80bdffc76f500c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:34.507461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:34.507831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:34.508219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:34.612113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] MFA Rules not processed for user `fa0e6b920dd24011ac80bdffc76f500c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:34.615669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:34.924958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 896/1758] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:34.935528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:34.943124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:34.969124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:34.970585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:35.082458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:35.132979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dcf0a-c0a4-41a8-a634-23891a67fc83 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:35.139332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ddb8-000e-461b-9dd5-8b91feb6f79d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:35.139994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:35.140958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:35.141218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:35.141620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:35.165575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:35.166579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:35.167289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:35.167630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:35.167895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:35.180375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:35.193506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:35.194990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:35.202721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 897/1759] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:35.223783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1760] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:35.363830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:35.365165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:35.372292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:35.410751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:35.423075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021baa4-9143-4dc8-b4d3-b497b0593139 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:35.448698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe40da-a67e-4a86-9cc8-3098c7bd1812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:35.448698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe40da-a67e-4a86-9cc8-3098c7bd1812 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:35.448698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe40da-a67e-4a86-9cc8-3098c7bd1812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:35.449028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe40da-a67e-4a86-9cc8-3098c7bd1812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:35.657127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40798c6b-6f03-4980-b993-85f0f62f3981 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:35.676520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:35.677225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:35.677520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:35.677807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] PATH_INFO: `/v3/projects/02b96e07705c48f7b3f675a90ca7d72d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:35.679133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b96e07705c48f7b3f675a90ca7d72d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:35.681190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:35.716493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe40da-a67e-4a86-9cc8-3098c7bd1812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:35.718012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe40da-a67e-4a86-9cc8-3098c7bd1812 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:35.750282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1761] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:35.803061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b96e07705c48f7b3f675a90ca7d72d', 'request_id': 'req-75cc7d3a-ede7-4030-8146-7ad250a038be'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:35.813848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b96e07705c48f7b3f675a90ca7d72d', 'request_id': 'req-75cc7d3a-ede7-4030-8146-7ad250a038be'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:35.822737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b96e07705c48f7b3f675a90ca7d72d', 'request_id': 'req-75cc7d3a-ede7-4030-8146-7ad250a038be'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:35.839659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] The token cache is being invalidate because project 02b96e07705c48f7b3f675a90ca7d72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:35.839903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:35.840833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75cc7d3a-ede7-4030-8146-7ad250a038be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:35.842909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 898/1762] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:35 2022] DELETE /identity/v3/projects/02b96e07705c48f7b3f675a90ca7d72d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:35.848672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:35.849361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:35.849878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:35.944154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:35.945560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:36.197759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1763] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:36.205345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:36.206329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:36.206876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:36.275799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:36.277093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:36.325306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 899/1764] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:36.325997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:36.365411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7724b1-f42f-4112-9646-a7f6a73d5920 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:36.383409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:36.384995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:36.385303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:36.385629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:36.389943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:36.390329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:36.395162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:36.428587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1765] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:36.434112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:36.448532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8f7dd-ee2f-406f-81bb-fa151e46a69f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:36.472742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:36.473448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:36.473739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:36.474028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:36.474740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:36.495088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:36.518295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:36.520566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:36.543649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5913b-8095-4524-bd6a-b1714ecf1a1c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:36.559997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:36.560934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:36.561360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:36.561778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] PATH_INFO: `/v3/users/96ff5b6366634481b30dabd37556fd48` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:36.572437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ff5b6366634481b30dabd37556fd48)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:36.577792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:36.582154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 900/1766] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:36 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:36.584490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:36.721375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ff5b6366634481b30dabd37556fd48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:36.722010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ff5b6366634481b30dabd37556fd48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:36.750990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ff5b6366634481b30dabd37556fd48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:36.752140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ff5b6366634481b30dabd37556fd48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:36.758764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:36.777506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-122f9c9e-dc6b-4b8b-9dbf-9eea122490f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ff5b6366634481b30dabd37556fd48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:36.798715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1767] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:36 2022] DELETE /identity/v3/users/96ff5b6366634481b30dabd37556fd48 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:36.806080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:36.806647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:36.808293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:36.860737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2431a-f003-4e85-b883-a5c289995b50 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:36.864348 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:36.877882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:36.904289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:37.057639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1768] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:37.066903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:37.068631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:37.070131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:37.070568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:37.073725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:37.074533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:37.076192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:37.077919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:37.172696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 901/1769] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:36 2022] POST /identity/v3/users => generated 282 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:37.172696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:37.175023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:37.217388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305db976-774d-4208-843b-fcd3e2beb376 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:37.252308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:37.253123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:37.253531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:37.253848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:37.269150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:37.283480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:37.371602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 902/1770] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:37.382754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ac4f-a419-4188-9eea-f8b3eec94180 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:37.384896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:37.402925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507700e5-9a7d-4629-82c6-fa664cea87bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:37.407169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507700e5-9a7d-4629-82c6-fa664cea87bb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:37.408594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507700e5-9a7d-4629-82c6-fa664cea87bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:37.408987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507700e5-9a7d-4629-82c6-fa664cea87bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:37.417458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311470a-da22-4406-bfa5-80ea31af249e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:37.432210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:37.432867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:37.433155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:37.433435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:37.434135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:37.448052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:37.468596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507700e5-9a7d-4629-82c6-fa664cea87bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:37.480029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 903/1771] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:37.491319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507700e5-9a7d-4629-82c6-fa664cea87bb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:37.506103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:37.528621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1772] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:37.564526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5308-b0e6-4719-8586-1ee1b880c962 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:37.587758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:37.588474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:37.588755 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:37.589037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] PATH_INFO: `/v3/projects/48d4360f22664800a1459ef241e385cb/users/c133cc1903384e19acc6b5ce281f282e/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:37.639173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d4360f22664800a1459ef241e385cb, user_id=c133cc1903384e19acc6b5ce281f282e, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:37.641909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:37.700572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 904/1773] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:37 2022] PUT /identity/v3/projects/48d4360f22664800a1459ef241e385cb/users/c133cc1903384e19acc6b5ce281f282e/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:37.710439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:37.710799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:37.712987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:37.845731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] MFA Rules not processed for user `c133cc1903384e19acc6b5ce281f282e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:37.852236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:38.073629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1774] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:37 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:38.079011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:38.224706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888b057-bfc5-4f18-85d8-264adb06f2f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:38.230995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:38.233743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:38.234480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:38.235006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:38.235983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:38.239117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:38.309244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:38.312590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 905/1775] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:38 2022] POST /identity/v3/projects => generated 363 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:38.332667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f09eb0-1d21-47c4-98de-a96a65558be3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:38.355579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:38.566041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:38.567555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:38.567856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:38.569027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:38.571963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:38.578849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:38.660862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1776] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:38 2022] POST /identity/v3/users => generated 288 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:38.666322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:38.683533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d169a9-714f-41da-8ebf-12f0497520b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:38.706450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:38.709516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:38.712266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:38.712829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:38.713833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:38.719718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:38.748857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 906/1777] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:38.754056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:38.783069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da56bb-269c-478a-8227-7cefd94107b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:38.809449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:38.810195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:38.810482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:38.810764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] PATH_INFO: `/v3/projects/9cddfce293f54027ac4399f067d2b6c3/users/1419fbc8f2db41d495d65f68df1121f7/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:38.850225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cddfce293f54027ac4399f067d2b6c3, user_id=1419fbc8f2db41d495d65f68df1121f7, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:38.853007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:38.888934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1778] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:38 2022] PUT /identity/v3/projects/9cddfce293f54027ac4399f067d2b6c3/users/1419fbc8f2db41d495d65f68df1121f7/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:38.891763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:38.918382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd706-be49-4c19-a5ad-241a91bfad9d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:39.048301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:39.139034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404d72d-6552-488f-9018-39addb20f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:39.139034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404d72d-6552-488f-9018-39addb20f4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:39.139034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404d72d-6552-488f-9018-39addb20f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:39.139034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404d72d-6552-488f-9018-39addb20f4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:39.139480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404d72d-6552-488f-9018-39addb20f4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:39.139480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404d72d-6552-488f-9018-39addb20f4bb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:39.167150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 907/1779] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:39.168838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:39.185320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83714ba8-eb13-4a62-9309-f96d4fb7a78e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:39.210144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:39.211114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:39.211581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:39.212014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] PATH_INFO: `/v3/projects/9cddfce293f54027ac4399f067d2b6c3/users/1419fbc8f2db41d495d65f68df1121f7/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:39.223535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cddfce293f54027ac4399f067d2b6c3, user_id=1419fbc8f2db41d495d65f68df1121f7, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:39.226616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:39.277005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1780] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:39 2022] PUT /identity/v3/projects/9cddfce293f54027ac4399f067d2b6c3/users/1419fbc8f2db41d495d65f68df1121f7/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:39.280216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:39.280747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:39.280988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:39.371022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] MFA Rules not processed for user `1419fbc8f2db41d495d65f68df1121f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:39.373150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:39.650463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 908/1781] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:39.671052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:39.673409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:39.808038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01686be1-5b31-4a17-8c47-a97d940babe5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:39.822898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:39.823766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:39.824175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:39.824580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:39.851321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:39.853163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:39.880508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1782] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:39 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:40.037949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:40.061360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a8d76-0433-4b64-b421-ba4cd812f374 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:40.068102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:40.069247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:40.071059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:40.071577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:40.279064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:40.280985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:40.336234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 909/1783] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:40 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:41.357840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:41.360662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:41.709632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4d9d-fe18-4228-93b3-3f1d9a97b4ff service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:41.723834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:41.724545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:41.724837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:41.725200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] PATH_INFO: `/v3/projects/c5d25eeaa5e346458230dfc09fa303ed` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:41.732284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d25eeaa5e346458230dfc09fa303ed)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:41.734257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:41.760501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:41.912001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d25eeaa5e346458230dfc09fa303ed', 'request_id': 'req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:41.948325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d25eeaa5e346458230dfc09fa303ed', 'request_id': 'req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:41.964517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d25eeaa5e346458230dfc09fa303ed', 'request_id': 'req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.012342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] The token cache is being invalidate because project c5d25eeaa5e346458230dfc09fa303ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:42.012809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.016916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-134cec5f-cfa1-481c-9aa1-1ce8a3d37187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.024698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1784] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:41 2022] DELETE /identity/v3/projects/c5d25eeaa5e346458230dfc09fa303ed => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:42.029584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:42.106522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cf1f0-dfb4-46ea-bb6c-824511185656 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:42.334052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:42.361601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:42.364078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:42.364854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:42.365523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] PATH_INFO: `/v3/users/fa0e6b920dd24011ac80bdffc76f500c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:42.367566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa0e6b920dd24011ac80bdffc76f500c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:42.370380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:42.492446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9f701-fab0-4e3c-af77-8e5a4c294c61 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:42.493398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9f701-fab0-4e3c-af77-8e5a4c294c61 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:42.493616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9f701-fab0-4e3c-af77-8e5a4c294c61 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:42.495049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1785] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:42.567767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa0e6b920dd24011ac80bdffc76f500c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.590838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa0e6b920dd24011ac80bdffc76f500c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.591793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa0e6b920dd24011ac80bdffc76f500c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.592891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa0e6b920dd24011ac80bdffc76f500c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.654708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d02e94ab-6b60-4c5e-9e3f-a0042072ab1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa0e6b920dd24011ac80bdffc76f500c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:42.675114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:42.675430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:42.675881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:42.703419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 910/1786] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:42 2022] DELETE /identity/v3/users/fa0e6b920dd24011ac80bdffc76f500c => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:42.734806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:42.736122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:42.959575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1787] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:21:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:43.296098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0625f-d0cc-4f95-85b9-d65b62e342a8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:43.296572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0625f-d0cc-4f95-85b9-d65b62e342a8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:43.297054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0625f-d0cc-4f95-85b9-d65b62e342a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:43.348234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b0b0625f-d0cc-4f95-85b9-d65b62e342a8 None None] MFA Rules not processed for user `c2db6ada97da4ee89f3ce8de98f2653a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:43.353219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b0b0625f-d0cc-4f95-85b9-d65b62e342a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:43.408656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:43.411756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:43.456074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da540cc1-51cb-4191-b5ee-8f1f33fa1295 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:43.485934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84100d09-6ec0-4775-927d-cafd09ff7ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:43.486871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100d09-6ec0-4775-927d-cafd09ff7ded service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:43.487327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100d09-6ec0-4775-927d-cafd09ff7ded service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:43.488363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100d09-6ec0-4775-927d-cafd09ff7ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:43.528251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84100d09-6ec0-4775-927d-cafd09ff7ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:43.533550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 911/1788] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:43.536565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84100d09-6ec0-4775-927d-cafd09ff7ded service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:43.580634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1789] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:43.962879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:43.963460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:43.963916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:44.005770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] MFA Rules not processed for user `af5a109a6fb94042aeacbf141d7322ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:44.007231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:44.138752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a56a3-8bb0-4c1a-b4cf-ad8061748adb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:44.139035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a56a3-8bb0-4c1a-b4cf-ad8061748adb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:44.139511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a56a3-8bb0-4c1a-b4cf-ad8061748adb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:44.141660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1790] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 05:21:44.176662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab98a80-95c4-43d5-8ccc-695a269ec18a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:44.176964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab98a80-95c4-43d5-8ccc-695a269ec18a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:44.177248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab98a80-95c4-43d5-8ccc-695a269ec18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:44.242613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dab98a80-95c4-43d5-8ccc-695a269ec18a None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:44.246934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 912/1791] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:44.266705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dab98a80-95c4-43d5-8ccc-695a269ec18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:44.333604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1792] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:21:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:44.553246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:44.554976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:44.616255 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:44.739404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39b50c-8f9d-4b57-9e6c-e38655000b4f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:44.744581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:44.745253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:44.745537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:44.745815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] PATH_INFO: `/v3/users/c133cc1903384e19acc6b5ce281f282e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:44.747005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c133cc1903384e19acc6b5ce281f282e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:44.749347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:44.829099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c133cc1903384e19acc6b5ce281f282e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:44.843200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c133cc1903384e19acc6b5ce281f282e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:44.849823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c133cc1903384e19acc6b5ce281f282e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:44.850710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c133cc1903384e19acc6b5ce281f282e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:44.914846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c133cc1903384e19acc6b5ce281f282e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:44.956825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 913/1793] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:44 2022] DELETE /identity/v3/users/c133cc1903384e19acc6b5ce281f282e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:44.958833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:44.959417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:44.959507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:45.001834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:45.003119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:45.137180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1794] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:45.218045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:45.220006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:45.308913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1b807-804a-4836-a456-0d98468227b2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:45.314057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:45.314895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:45.317538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:45.317930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:45.333845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:45.335476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:45.359337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 914/1795] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:45.962900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:45.966578 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.002181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05a812-8f30-4952-9713-f7708b3102b0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:46.009047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:46.010103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.010495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.010849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] PATH_INFO: `/v3/projects/48d4360f22664800a1459ef241e385cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.012465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d4360f22664800a1459ef241e385cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:46.014774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:46.017726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:46.025135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:46.026808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.050023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c413607-de3b-4a54-8086-6918dd99a9de service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:46.062016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:46.062913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.063381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.063822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] PATH_INFO: `/v3/projects/6f4c8861e1094021bf58874d58a5a132` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.067227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4c8861e1094021bf58874d58a5a132)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:46.069508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:46.166372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:46.177925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d4360f22664800a1459ef241e385cb', 'request_id': 'req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.188158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d4360f22664800a1459ef241e385cb', 'request_id': 'req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.189466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4c8861e1094021bf58874d58a5a132', 'request_id': 'req-d12ee0cd-15cb-45fe-9124-fca4d4232741'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.204380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4c8861e1094021bf58874d58a5a132', 'request_id': 'req-d12ee0cd-15cb-45fe-9124-fca4d4232741'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.205888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d4360f22664800a1459ef241e385cb', 'request_id': 'req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.214430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4c8861e1094021bf58874d58a5a132', 'request_id': 'req-d12ee0cd-15cb-45fe-9124-fca4d4232741'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.231830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] The token cache is being invalidate because project 48d4360f22664800a1459ef241e385cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:46.232390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.233286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987d902d-3f2d-4f4f-aa67-f7b6b125d4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.234247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] The token cache is being invalidate because project 6f4c8861e1094021bf58874d58a5a132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:46.234605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.240470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.240787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1796] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:45 2022] DELETE /identity/v3/projects/48d4360f22664800a1459ef241e385cb => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:46.241847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 915/1797] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:46 2022] DELETE /identity/v3/projects/6f4c8861e1094021bf58874d58a5a132 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:46.242915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:46.248304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.248644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.248932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.249076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.305577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:46.307173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.389152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ee0cd-15cb-45fe-9124-fca4d4232741 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:46.397476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:46.398321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.398679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.399031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] PATH_INFO: `/v3/users/1419fbc8f2db41d495d65f68df1121f7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.401568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1419fbc8f2db41d495d65f68df1121f7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:46.402638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1798] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:46.403649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:46.404911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:46.405025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.515174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1419fbc8f2db41d495d65f68df1121f7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.530886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4d2c9-f188-4369-8c14-fc4ecf9a2c09 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:46.552710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1419fbc8f2db41d495d65f68df1121f7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.554332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1419fbc8f2db41d495d65f68df1121f7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.557480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1419fbc8f2db41d495d65f68df1121f7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.564545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:46.565652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.566015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.566309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.572360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.591560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:46.595557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe322a07-1221-4996-922d-2658e7aaf226 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:46.599046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1419fbc8f2db41d495d65f68df1121f7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:46.624924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1799] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:46.626741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.627383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.627864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.642409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 916/1800] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:46 2022] DELETE /identity/v3/users/1419fbc8f2db41d495d65f68df1121f7 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:46.665950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:46.668179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:46.684267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:46.824101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1801] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:46.824900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:46.837383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f0eb9-a7bc-4d31-ac56-7a34f020b897 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:46.868598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:46.869372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.869683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.870082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.871098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:46.871467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:46.873193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:46.929425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:46.936057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 917/1802] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:21:46.943250 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f957b2-482d-4002-b951-e8b77394dccd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:46.957245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:46.958281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:46.959400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:46.959981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:46.960940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:46.963411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.031290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.055535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1803] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:46 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:47.061298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad52645-56ae-424d-a755-1b3598aa5e22 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:47.133582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:47.175885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:47.176656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.176949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.177900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.178185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:47.180250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.237085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 918/1804] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:47 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:47.242676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.256814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cce5-6cc5-4cab-b519-1c7b5219864a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:47.263184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:47.264029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.264188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.264638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.265548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:47.269799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.282705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1805] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:47.287819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.302141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139268dd-fd85-4660-b1bc-00e1b5afa054 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:47.310120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabeb21-d673-44f0-a59c-62c6dc836b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:47.310815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabeb21-d673-44f0-a59c-62c6dc836b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.311098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabeb21-d673-44f0-a59c-62c6dc836b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.311409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabeb21-d673-44f0-a59c-62c6dc836b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.312284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabeb21-d673-44f0-a59c-62c6dc836b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:47.313922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabeb21-d673-44f0-a59c-62c6dc836b80 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.328725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 919/1806] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:47.329993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.342349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6c45c-3bc3-4544-9f2e-a6c560444105 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:47.348590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:47.349428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.349813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.350235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] PATH_INFO: `/v3/projects/89cde474792142f08bf7848d5a749488/users/d0e67b37a7564114bf61995f8a03d48d/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.376511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cde474792142f08bf7848d5a749488, user_id=d0e67b37a7564114bf61995f8a03d48d, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:47.381763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.404282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1807] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:47 2022] PUT /identity/v3/projects/89cde474792142f08bf7848d5a749488/users/d0e67b37a7564114bf61995f8a03d48d/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:47.416649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.416900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.417193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.473504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] MFA Rules not processed for user `d0e67b37a7564114bf61995f8a03d48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:47.478264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:47.478751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.480998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:47.582334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45785c-76f2-4a2e-8594-6fc295792e73 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:47.589073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:47.590558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.591013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.591511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] PATH_INFO: `/v3/projects/9cddfce293f54027ac4399f067d2b6c3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.593013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cddfce293f54027ac4399f067d2b6c3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:47.595129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.608992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 920/1808] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:47.617250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.618803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:47.698806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cddfce293f54027ac4399f067d2b6c3', 'request_id': 'req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:47.706567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219e790-92e6-4629-a8de-0ca8c9b2c23f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:47.706863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cddfce293f54027ac4399f067d2b6c3', 'request_id': 'req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:47.715405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:47.719074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cddfce293f54027ac4399f067d2b6c3', 'request_id': 'req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:47.750310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] The token cache is being invalidate because project 9cddfce293f54027ac4399f067d2b6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:47.750600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:47.751302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7457bb7-1f2e-4dc0-ba4e-d0c5ce2de922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:47.758671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1809] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:47 2022] DELETE /identity/v3/projects/9cddfce293f54027ac4399f067d2b6c3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:47.761394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109476bf-e480-4dc4-a90f-7159e5ccac80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.762477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109476bf-e480-4dc4-a90f-7159e5ccac80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.762804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109476bf-e480-4dc4-a90f-7159e5ccac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.791577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:47.792586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.792930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.793272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.795387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:47.800399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-109476bf-e480-4dc4-a90f-7159e5ccac80 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:47.801835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-109476bf-e480-4dc4-a90f-7159e5ccac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:47.807176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:47.912058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:47.913983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:47.958956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1810] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:47.961283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 921/1811] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:47.967282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:47.967872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:47.968057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:47.991468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:47.992978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:48.016866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f6e6d-15a0-48fc-8595-dfc98714da35 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.029913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.030655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.030951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.031242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] PATH_INFO: `/v3/users/d0e67b37a7564114bf61995f8a03d48d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.036769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e67b37a7564114bf61995f8a03d48d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.039040 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.039522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:48.041958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:48.101842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1812] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:48.103286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.185509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d73f6-f223-40f6-aaeb-1bec06f86306 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.208648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.209326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.209615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.209927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.210997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:48.210997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.222114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.229192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e67b37a7564114bf61995f8a03d48d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:48.249745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e67b37a7564114bf61995f8a03d48d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:48.250605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e67b37a7564114bf61995f8a03d48d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:48.253802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e67b37a7564114bf61995f8a03d48d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:48.270366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1813] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:48.271745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.278325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:48.281825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d018c9b-28d5-4bb4-97c3-a440c868860d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e67b37a7564114bf61995f8a03d48d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:48.358871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 922/1814] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:47 2022] DELETE /identity/v3/users/d0e67b37a7564114bf61995f8a03d48d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:48.359526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d71a-6584-47dc-aba3-a66258335f6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.379283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.379716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.383266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.415857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:48.418356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:48.470848 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.472668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.472970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.473241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.473942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.478147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.543608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 923/1815] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:48.547691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1816] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:48 2022] POST /identity/v3/projects => generated 369 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:48.552714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.556196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.558589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:48.574418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446509-8b99-4a88-b33d-7048bfb53c87 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.605579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:48.692853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0224a-1036-407d-8894-0756f39c95b6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.699085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.700010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.700356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.700645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.715698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:48.717046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.718018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.718416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.718706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.719485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.723009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.761977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 924/1817] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:48 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:48.762233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.802037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513cf34-301d-4c6d-a51f-e5fc0481a644 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.808277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.809132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.809332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.809767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.810511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.812285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.819624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.821101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.826276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 925/1818] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:48.827144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.831317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:48.850825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b804e63-8eff-4dfd-9486-f10cf32e059e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.857761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.858923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1819] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:48.859039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.859672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.872963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.873553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] PATH_INFO: `/v3/users/ba8749223fd44b29906737989dc22438` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.877283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8749223fd44b29906737989dc22438)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.883237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4cbab-3a5b-4f0a-9e9f-7fdc8236d476 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.884454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.897112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.897765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.898331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.898331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.899059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.900728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:48.937155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1820] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:48.947126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:48.960761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304f996-0813-4d4e-ba03-1249c7064b02 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:48.967282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6302fc7-f071-4846-9c86-9960336effd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:48.968067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6302fc7-f071-4846-9c86-9960336effd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:48.968067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6302fc7-f071-4846-9c86-9960336effd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:48.968218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6302fc7-f071-4846-9c86-9960336effd2 admin admin] PATH_INFO: `/v3/projects/b9998a3ac8144e31bcd744705f280b22/users/0e5b0e2162b94df282e01b802538d8fb/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:48.991402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6302fc7-f071-4846-9c86-9960336effd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9998a3ac8144e31bcd744705f280b22, user_id=0e5b0e2162b94df282e01b802538d8fb, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:48.994709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6302fc7-f071-4846-9c86-9960336effd2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:49.002486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8749223fd44b29906737989dc22438'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:49.021983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8749223fd44b29906737989dc22438'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:49.022867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8749223fd44b29906737989dc22438'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:49.023964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8749223fd44b29906737989dc22438'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:49.045473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1821] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:48 2022] PUT /identity/v3/projects/b9998a3ac8144e31bcd744705f280b22/users/0e5b0e2162b94df282e01b802538d8fb/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:49.059807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.060104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.060385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.065808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b744da93-3155-4d4c-85f4-fa3bbdb89cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8749223fd44b29906737989dc22438'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:49.107295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 926/1822] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:48 2022] DELETE /identity/v3/users/ba8749223fd44b29906737989dc22438 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:49.111613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] MFA Rules not processed for user `0e5b0e2162b94df282e01b802538d8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:49.114164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.114466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.114889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.117029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:49.145106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:49.149024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:49.318609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 927/1823] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:49.326634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1824] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:49.330166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:49.331914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:49.353411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bafa4-0c06-41ed-b61a-e95a66b7aff0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:49.360648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:49.361620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.362382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.363540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.364459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:49.366862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:49.383606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:49.421685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 928/1825] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:49 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:49.423136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:49.448147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:49.449983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:49.453656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74535d51-4100-4a54-b4fb-abf9377bc1ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:49.589981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec003c0-149b-4af7-99fd-8fc4eb3bcca3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:49.595880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:49.597820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.598295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.598719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.636063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:49.636818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.637176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.637589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.638521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:49.642571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:49.660459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:49.681202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1826] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:49 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:49.685409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:49.696480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38848938-561b-4533-962d-7b640d26a641 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:49.702857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:49.703552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.703844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.704126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.704840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:49.707538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:49.736469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1827] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:49.741997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:49.820499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:49.821981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:49.830552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9e79c-af39-461c-8da6-51a1ae6aeebd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:49.833503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 929/1828] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:49.834357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:49.848784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:49.927911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11088b-293a-4cf8-834a-edc075d22922 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:49.929284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:49.936799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.938463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.938795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] PATH_INFO: `/v3/projects/5be2087b98e24dd4b7abfb264e6022f4/users/2e2e0e64a888437a8272120d9a8b576f/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.962384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:49.965649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:49.966488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:49.967194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] PATH_INFO: `/v3/projects/89cde474792142f08bf7848d5a749488` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:49.969235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89cde474792142f08bf7848d5a749488)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:49.971780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.019787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be2087b98e24dd4b7abfb264e6022f4, user_id=2e2e0e64a888437a8272120d9a8b576f, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.025988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.059443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1829] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:49 2022] PUT /identity/v3/projects/5be2087b98e24dd4b7abfb264e6022f4/users/2e2e0e64a888437a8272120d9a8b576f/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:50.063897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.109747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cde474792142f08bf7848d5a749488', 'request_id': 'req-66c65be2-fe54-4917-8532-e4cf7a55f6d9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:50.150090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cde474792142f08bf7848d5a749488', 'request_id': 'req-66c65be2-fe54-4917-8532-e4cf7a55f6d9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:50.165884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cde474792142f08bf7848d5a749488', 'request_id': 'req-66c65be2-fe54-4917-8532-e4cf7a55f6d9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:50.201156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] The token cache is being invalidate because project 89cde474792142f08bf7848d5a749488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:50.202784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:50.203691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66c65be2-fe54-4917-8532-e4cf7a55f6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:50.209926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 930/1830] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:49 2022] DELETE /identity/v3/projects/89cde474792142f08bf7848d5a749488 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:50.215870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859d91b-1923-4fca-8c07-5d2e28ed6a5a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.216059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859d91b-1923-4fca-8c07-5d2e28ed6a5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.217584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859d91b-1923-4fca-8c07-5d2e28ed6a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.224960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7a25-4b78-4807-ad73-67a3efba9ff0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.256575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2859d91b-1923-4fca-8c07-5d2e28ed6a5a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:50.257821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2859d91b-1923-4fca-8c07-5d2e28ed6a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:50.285027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:50.388351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 931/1831] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:50.390909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.391397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.391827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.399592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:50.400252 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.400540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.400824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.407592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.412410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.424588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:50.426184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:50.443255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1832] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:50.443518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 932/1833] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:50.444509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.446447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.448808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:50.467716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03d4f-ae6a-4270-80c6-b0e37e642cc6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.469060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f6b91-26b1-44c0-951c-02f2ad46db63 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.475656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:50.476548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.477017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.477457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.478394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:50.478783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.481960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.483841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec74b24-4e59-43e7-8cb7-a0b391e20e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:50.484622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec74b24-4e59-43e7-8cb7-a0b391e20e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.484911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec74b24-4e59-43e7-8cb7-a0b391e20e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.485288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec74b24-4e59-43e7-8cb7-a0b391e20e27 admin admin] PATH_INFO: `/v3/projects/5be2087b98e24dd4b7abfb264e6022f4/users/2e2e0e64a888437a8272120d9a8b576f/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.492343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec74b24-4e59-43e7-8cb7-a0b391e20e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be2087b98e24dd4b7abfb264e6022f4, user_id=2e2e0e64a888437a8272120d9a8b576f, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.495291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec74b24-4e59-43e7-8cb7-a0b391e20e27 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.501554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 933/1834] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:50.510046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.530399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874f185-e036-484e-a423-5c951990af78 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.546566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1835] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:50 2022] PUT /identity/v3/projects/5be2087b98e24dd4b7abfb264e6022f4/users/2e2e0e64a888437a8272120d9a8b576f/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:50.551682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.552086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.552503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.601502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] MFA Rules not processed for user `2e2e0e64a888437a8272120d9a8b576f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:50.603097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:50.620558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:50.621202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.621488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.621768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.622452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.624876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.655371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:50.686876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 934/1836] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:50 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:50.691619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.747475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1837] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:50.766198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.769647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:50.797287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af7ae-a2f3-4a03-a14c-2d47d6c52ade admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.805002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:50.806078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.806940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.807240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.807955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.810726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:50.857430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 935/1838] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:50 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:50.857816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:50.866795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7f28b-7e10-4c5b-93d1-0e791ae17f62 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.887256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607fed-d151-4c8e-bb45-592fd2ef069e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:50.931839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:50.933109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:50.933109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:50.934551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:50.937027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:50.941818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:51.002848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 936/1839] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:51.010104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:51.029246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:51.031614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:51.032082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:51.032560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:51.038711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36a338-995e-4d18-9161-1c4200f0abc9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:51.053041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:51.054875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:51.054875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:51.054875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:51.055335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:51.058688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:51.069301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:51.084352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 937/1840] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:51.089826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:51.099936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:51.114770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051204af-c015-40da-b73d-db28213c74a3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:51.123044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:51.123044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:51.123044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:51.123044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] PATH_INFO: `/v3/projects/a67a3b58e8c04374abdcd8fec4e0b411` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:51.125438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a67a3b58e8c04374abdcd8fec4e0b411)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:51.129539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:51.179852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:51.181342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:51.201713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1841] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:51.203962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:51.224090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd787c5-e174-4dbe-9b91-106b9a5bf57c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:51.224979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a67a3b58e8c04374abdcd8fec4e0b411', 'request_id': 'req-6c89aac8-2d6e-442f-9020-76b295f346e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.239469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a67a3b58e8c04374abdcd8fec4e0b411', 'request_id': 'req-6c89aac8-2d6e-442f-9020-76b295f346e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.249145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a67a3b58e8c04374abdcd8fec4e0b411', 'request_id': 'req-6c89aac8-2d6e-442f-9020-76b295f346e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.297013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] The token cache is being invalidate because project a67a3b58e8c04374abdcd8fec4e0b411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:51.297642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.300893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.305334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 938/1842] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:51 2022] DELETE /identity/v3/projects/a67a3b58e8c04374abdcd8fec4e0b411 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:51.309898 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:51.314859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:51.348795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:51.445026 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ba437-26cb-4545-9404-9a86faec91c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:51.449411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ba437-26cb-4545-9404-9a86faec91c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:51.449937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ba437-26cb-4545-9404-9a86faec91c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:51.450443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ba437-26cb-4545-9404-9a86faec91c2 admin admin] PATH_INFO: `/v3/projects/02466603267e4f8b8bbc7ca347b09124/users/76273fd3588d45be816d2c57636f3226/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:51.462256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89aac8-2d6e-442f-9020-76b295f346e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:51.472734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:51.474045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:51.474700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:51.475089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] PATH_INFO: `/v3/users/f8847b3ba3e6409d866165e6e9fd421b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:51.476792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8847b3ba3e6409d866165e6e9fd421b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:51.479810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:51.511128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ba437-26cb-4545-9404-9a86faec91c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02466603267e4f8b8bbc7ca347b09124, user_id=76273fd3588d45be816d2c57636f3226, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:51.519299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ba437-26cb-4545-9404-9a86faec91c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:51.548138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1843] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:51 2022] PUT /identity/v3/projects/02466603267e4f8b8bbc7ca347b09124/users/76273fd3588d45be816d2c57636f3226/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:51.566024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1614-1f52-49ab-b9d2-dd9677152240 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:51.566328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1614-1f52-49ab-b9d2-dd9677152240 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:51.566614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1614-1f52-49ab-b9d2-dd9677152240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:51.596883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8847b3ba3e6409d866165e6e9fd421b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.609227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8847b3ba3e6409d866165e6e9fd421b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.610350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8847b3ba3e6409d866165e6e9fd421b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.613790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8847b3ba3e6409d866165e6e9fd421b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.646988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8847b3ba3e6409d866165e6e9fd421b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:51.666368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:51.680617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 939/1844] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:51 2022] DELETE /identity/v3/users/f8847b3ba3e6409d866165e6e9fd421b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:51.698011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c7ef1614-1f52-49ab-b9d2-dd9677152240 None None] MFA Rules not processed for user `76273fd3588d45be816d2c57636f3226`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:51.699274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c7ef1614-1f52-49ab-b9d2-dd9677152240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:51.857979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1845] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:51.871935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:51.874760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:52.031075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f3309-d1a1-4f10-b0b8-b015762dc1c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:52.038986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8b000-eeeb-49f5-bc77-cb30b3dd21d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:52.040017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8b000-eeeb-49f5-bc77-cb30b3dd21d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:52.040492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8b000-eeeb-49f5-bc77-cb30b3dd21d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:52.041260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8b000-eeeb-49f5-bc77-cb30b3dd21d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:52.058642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8b000-eeeb-49f5-bc77-cb30b3dd21d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:52.063180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8b000-eeeb-49f5-bc77-cb30b3dd21d9 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:52.087506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 940/1846] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:52.939198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678ce0-18c3-4ecd-bbf6-26c2c4a7e1c7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:52.942471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678ce0-18c3-4ecd-bbf6-26c2c4a7e1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:52.942471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678ce0-18c3-4ecd-bbf6-26c2c4a7e1c7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:52.942471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1847] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:21:52.979780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:52.980304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:52.980304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:53.044289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:53.046559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:53.289637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 941/1848] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:21:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:53.330355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678ce0-18c3-4ecd-bbf6-26c2c4a7e1c7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:53.336670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6678ce0-18c3-4ecd-bbf6-26c2c4a7e1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:53.549227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:53.550918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:53.558885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678ce0-18c3-4ecd-bbf6-26c2c4a7e1c7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:53.565448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:53.566386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:53.566806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:53.567467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] PATH_INFO: `/v3/projects/85eb103fb0934fffa765d31b0c8808e4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:53.568988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85eb103fb0934fffa765d31b0c8808e4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:53.571072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:53.574787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aeab71-d980-4a8f-81f1-a7fa5f268980 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:53.592173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4577500-a12b-435f-b491-baff5cb9018e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:53.593098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4577500-a12b-435f-b491-baff5cb9018e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:53.593497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4577500-a12b-435f-b491-baff5cb9018e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:53.594237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4577500-a12b-435f-b491-baff5cb9018e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:53.617912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4577500-a12b-435f-b491-baff5cb9018e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:53.622159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4577500-a12b-435f-b491-baff5cb9018e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:53.638069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85eb103fb0934fffa765d31b0c8808e4', 'request_id': 'req-755819d1-49d3-4348-97c3-9d302e5f4599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:53.640019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:53.649890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85eb103fb0934fffa765d31b0c8808e4', 'request_id': 'req-755819d1-49d3-4348-97c3-9d302e5f4599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:53.655628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 942/1849] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:53.681176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85eb103fb0934fffa765d31b0c8808e4', 'request_id': 'req-755819d1-49d3-4348-97c3-9d302e5f4599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:53.716466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] The token cache is being invalidate because project 85eb103fb0934fffa765d31b0c8808e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:53.717517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:53.719714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:53.724448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1850] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:53 2022] DELETE /identity/v3/projects/85eb103fb0934fffa765d31b0c8808e4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:53.730135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677732fb-e855-4d2f-a9ce-a0953e567713 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:53.730958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677732fb-e855-4d2f-a9ce-a0953e567713 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:53.730958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677732fb-e855-4d2f-a9ce-a0953e567713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:53.763604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-677732fb-e855-4d2f-a9ce-a0953e567713 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:53.765105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-677732fb-e855-4d2f-a9ce-a0953e567713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:53.920349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:53.922703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:53.985229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 943/1851] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:53.993357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:53.993712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:53.993999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.041146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:54.043683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:54.065465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 944/1852] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:54.069900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.094946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755819d1-49d3-4348-97c3-9d302e5f4599 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.110174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.110915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.111234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.111549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] PATH_INFO: `/v3/users/0e5b0e2162b94df282e01b802538d8fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.114112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0c7d6-ec45-4918-9adf-bd269eabb732 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.117073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e5b0e2162b94df282e01b802538d8fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.120029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.126536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.127272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.127635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.128673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.130416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:54.130736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.132516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.155012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 945/1853] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:21:54.156545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.162765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:54.169160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80da8a6-9003-411c-b65a-cef585147f07 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.177330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.178480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.178715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.179067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.179951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.182166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.218568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e5b0e2162b94df282e01b802538d8fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:54.219482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e5b0e2162b94df282e01b802538d8fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:54.223410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 946/1854] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:54 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:54.229279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e5b0e2162b94df282e01b802538d8fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:54.230023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e5b0e2162b94df282e01b802538d8fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:54.232281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.236088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:54.248183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4335b3a1-57e2-4868-b361-38076591f4a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e5b0e2162b94df282e01b802538d8fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:54.270561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1855] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:53 2022] DELETE /identity/v3/users/0e5b0e2162b94df282e01b802538d8fb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:54.275948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.276951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.277492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.305032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:54.306523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:54.358593 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708bdaf-5ede-4daa-b5d7-77c47cabd5dd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.375387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:54.391953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1856] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:54.402510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.406129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:54.452843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.454268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.454565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.454846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.456576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.459153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.484907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ee5b6-93b2-4cb5-99a7-ea1193a9c304 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.495155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.495949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.498358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.498746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.499733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 947/1857] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:54 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:54.505842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.521306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.523470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.527742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8e16-4345-4560-b485-383b826b586e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.532970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.534253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.534700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.535079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.536196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.537796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:54.538404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.555817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 948/1859] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:54.557066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.568544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68074d1f-467e-4ab0-a356-09aab6faf7df service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.574894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faf964-f449-4a30-8c8b-383c17b6dcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.575612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faf964-f449-4a30-8c8b-383c17b6dcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.576059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faf964-f449-4a30-8c8b-383c17b6dcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.576353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faf964-f449-4a30-8c8b-383c17b6dcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.577067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faf964-f449-4a30-8c8b-383c17b6dcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.578789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faf964-f449-4a30-8c8b-383c17b6dcc9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.594813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1860] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:54.597498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.614319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefa569-423e-4207-b4ef-32c34d6b218f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.638531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.639509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.640174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.640617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] PATH_INFO: `/v3/projects/5c5f38cd664246748d2a93e4133a928b/users/1048371bc1c64f15b87e1462b16fdd3f/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.667017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5f38cd664246748d2a93e4133a928b, user_id=1048371bc1c64f15b87e1462b16fdd3f, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.671455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.700859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 949/1861] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:54 2022] PUT /identity/v3/projects/5c5f38cd664246748d2a93e4133a928b/users/1048371bc1c64f15b87e1462b16fdd3f/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:54.716876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.717211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.717481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.751229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] MFA Rules not processed for user `1048371bc1c64f15b87e1462b16fdd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:54.753419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:54.847065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1862] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:54.849046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.908187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:54.932480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811044f9-3d6c-4abc-a45a-d5e7bd03cfb6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:54.940388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:54.941657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:54.942022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:54.942320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:54.943007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:54.945631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:54.987467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:54.995504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 950/1863] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:54 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:55.001641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaaa48-c9a2-471a-b720-d89850dbef1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.094629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.095335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.095870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.096030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.097659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.110646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.133279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1864] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:54 2022] POST /identity/v3/users => generated 297 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:55.137502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.152890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abc6bd-c741-40a1-b13a-492c02a76761 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.160291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.160836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.164994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.164994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.164994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.164994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.167174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.178017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:55.196316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550cfff-fe22-441e-ad8a-8761d5526f5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.204800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.205565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.205799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.206090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] PATH_INFO: `/v3/projects/b9998a3ac8144e31bcd744705f280b22` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.207642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9998a3ac8144e31bcd744705f280b22)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.209611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 951/1865] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:55.210667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.210782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.228726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:55.247621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24569e-6669-4bcf-8eb4-515f7fc0dbea admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.260936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.261638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.261920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.262201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] PATH_INFO: `/v3/projects/3bf975a185064fbeaba8486ada433e1c/users/1979a6aa5d0041418d22060c87886d06/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.285175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf975a185064fbeaba8486ada433e1c, user_id=1979a6aa5d0041418d22060c87886d06, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.291127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.302984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9998a3ac8144e31bcd744705f280b22', 'request_id': 'req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.311650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9998a3ac8144e31bcd744705f280b22', 'request_id': 'req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.321602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9998a3ac8144e31bcd744705f280b22', 'request_id': 'req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.354560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 952/1866] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:55 2022] PUT /identity/v3/projects/3bf975a185064fbeaba8486ada433e1c/users/1979a6aa5d0041418d22060c87886d06/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:55.363654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.371903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] The token cache is being invalidate because project b9998a3ac8144e31bcd744705f280b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:55.372317 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.376478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.385353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1867] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:55 2022] DELETE /identity/v3/projects/b9998a3ac8144e31bcd744705f280b22 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:55.386381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.391094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:55.476155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcd4d4-31e5-4d84-a98a-e1ae8b06e93f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.498267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513f95c-332b-43af-b67f-987e59c870f8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.512550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.515918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.516253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.516537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.517285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.520454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.537384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 953/1868] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:55.540072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.543152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:55.563247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce18f-be2e-4dae-9804-0699a94d43b1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.569658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7964b42-32fd-44a6-9cd6-d8f19234ee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.570510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7964b42-32fd-44a6-9cd6-d8f19234ee7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.570802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7964b42-32fd-44a6-9cd6-d8f19234ee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.571086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7964b42-32fd-44a6-9cd6-d8f19234ee7c admin admin] PATH_INFO: `/v3/projects/3bf975a185064fbeaba8486ada433e1c/users/1979a6aa5d0041418d22060c87886d06/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.572367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.573131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7964b42-32fd-44a6-9cd6-d8f19234ee7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf975a185064fbeaba8486ada433e1c, user_id=1979a6aa5d0041418d22060c87886d06, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.573454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.573929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.574880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] PATH_INFO: `/v3/users/2e2e0e64a888437a8272120d9a8b576f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.575871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7964b42-32fd-44a6-9cd6-d8f19234ee7c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.576738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2e0e64a888437a8272120d9a8b576f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:55.579138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:55.594658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 954/1869] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:55 2022] PUT /identity/v3/projects/3bf975a185064fbeaba8486ada433e1c/users/1979a6aa5d0041418d22060c87886d06/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:55.605711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.605992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.606325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:55.635850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:55.642562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2e0e64a888437a8272120d9a8b576f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.643666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2e0e64a888437a8272120d9a8b576f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.652695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2e0e64a888437a8272120d9a8b576f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.653367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2e0e64a888437a8272120d9a8b576f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.660466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] MFA Rules not processed for user `1979a6aa5d0041418d22060c87886d06`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:55.661789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:55.669045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2e0e64a888437a8272120d9a8b576f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:55.709130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1870] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:55 2022] DELETE /identity/v3/users/2e2e0e64a888437a8272120d9a8b576f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:55.796510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:55.813858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 955/1871] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:55.824696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:55.826676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:55.911327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5b11e-5a4e-4c0c-a9e0-639bf2b3768b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:55.915576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9ac38-734b-4310-b3ed-a759ccf1265b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:55.916475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9ac38-734b-4310-b3ed-a759ccf1265b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:55.916793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9ac38-734b-4310-b3ed-a759ccf1265b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:55.917128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9ac38-734b-4310-b3ed-a759ccf1265b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.008000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9ac38-734b-4310-b3ed-a759ccf1265b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.010131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9ac38-734b-4310-b3ed-a759ccf1265b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.039838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1872] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:56.184540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.186027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:56.271238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38184cdb-309b-4d36-bf79-7dd071e5f7e7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.338042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.338707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.338992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.339321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] PATH_INFO: `/v3/projects/5be2087b98e24dd4b7abfb264e6022f4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.340663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be2087b98e24dd4b7abfb264e6022f4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.342791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.412786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be2087b98e24dd4b7abfb264e6022f4', 'request_id': 'req-f66a43c8-0953-402c-962e-495235a92ac7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.418771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be2087b98e24dd4b7abfb264e6022f4', 'request_id': 'req-f66a43c8-0953-402c-962e-495235a92ac7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.425773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be2087b98e24dd4b7abfb264e6022f4', 'request_id': 'req-f66a43c8-0953-402c-962e-495235a92ac7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.438315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] The token cache is being invalidate because project 5be2087b98e24dd4b7abfb264e6022f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:56.438657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.439301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.445060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 956/1873] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:56 2022] DELETE /identity/v3/projects/5be2087b98e24dd4b7abfb264e6022f4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:56.446579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44dc0b-f6a9-4f78-a7ed-c6a543061711 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.447095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44dc0b-f6a9-4f78-a7ed-c6a543061711 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.447188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44dc0b-f6a9-4f78-a7ed-c6a543061711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.460191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.462643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:56.482245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2a44dc0b-f6a9-4f78-a7ed-c6a543061711 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:56.495932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a44dc0b-f6a9-4f78-a7ed-c6a543061711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:56.516954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:56.546232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a43c8-0953-402c-962e-495235a92ac7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.551971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1979a6aa5d0041418d22060c87886d06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bf975a185064fbeaba8486ada433e1c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.552614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.552913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.553198 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.553907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:56.554244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.555894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.576840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 957/1874] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:56.582226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.593930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1875] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:56.595280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc211a3-0643-4618-8fb1-646aaeadbb21 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.604083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.604588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.604907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.605956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1979a6aa5d0041418d22060c87886d06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bf975a185064fbeaba8486ada433e1c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.606709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.607246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.607492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.608277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.610758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.626991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:56.629710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:56.639527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1876] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:56.644386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 958/1877] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:56 2022] POST /identity/v3/projects => generated 355 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:56.644626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.648108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.649260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:56.655336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8149292-8a1f-4d84-ba91-102c57544b2a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.719478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.720115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.720403 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.720682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.721984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:56.722402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.724064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.724405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933d689-31e9-4909-9d02-5ceccb667548 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.729103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1979a6aa5d0041418d22060c87886d06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bf975a185064fbeaba8486ada433e1c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.729727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.730017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.730293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.730987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:56.731633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.732932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.736224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1878] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:21:56.741299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.746845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 959/1879] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:21:56.751615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bce85-7f7d-494b-a06d-4ac0add8dc4f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.756526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.757175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.757463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.757743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.758449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.760670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.791403 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1880] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:56 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:56.794054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.794800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.806212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31889c-4af9-4532-954d-3f8c0e0e0a91 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.861035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6293-01de-468e-8d78-96bbd6139241 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.865202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.865843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.866135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.866376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1979a6aa5d0041418d22060c87886d06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bf975a185064fbeaba8486ada433e1c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.866497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.867055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.867158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.867342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.867648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] PATH_INFO: `/v3/projects/a9dd96bc96af4f81845c3dcff1b812b9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.868798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a9dd96bc96af4f81845c3dcff1b812b9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:56.869058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.870734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:56.902517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 960/1881] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:56 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:21:56.903775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:56.915773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a63b2-c58e-41d5-a193-be8d8938f167 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:56.929311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9dd96bc96af4f81845c3dcff1b812b9', 'request_id': 'req-fc3e2959-eb5c-4ed6-b1bc-787388093f04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.936561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9dd96bc96af4f81845c3dcff1b812b9', 'request_id': 'req-fc3e2959-eb5c-4ed6-b1bc-787388093f04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.945815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9dd96bc96af4f81845c3dcff1b812b9', 'request_id': 'req-fc3e2959-eb5c-4ed6-b1bc-787388093f04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.956934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:56.961245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] The token cache is being invalidate because project a9dd96bc96af4f81845c3dcff1b812b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:56.962886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.963469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:56.965197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1882] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:56 2022] DELETE /identity/v3/projects/a9dd96bc96af4f81845c3dcff1b812b9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:56.997039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:56.997790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:56.998148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:56.998651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:56.999487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:57.001214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:57.013511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 961/1883] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:57.017659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:57.019652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:57.029547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:57.035596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e2959-eb5c-4ed6-b1bc-787388093f04 tempest-VolumeQuotaClassesTest-1338278271 tempest-VolumeQuotaClassesTest-1338278271-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:57.042505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67552b0c-5fa0-409b-a554-fca162ee720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:57.043386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67552b0c-5fa0-409b-a554-fca162ee720f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:57.043807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67552b0c-5fa0-409b-a554-fca162ee720f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:57.044195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67552b0c-5fa0-409b-a554-fca162ee720f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:57.045155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67552b0c-5fa0-409b-a554-fca162ee720f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:57.063024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67552b0c-5fa0-409b-a554-fca162ee720f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:57.105882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1884] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:21:57.111195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:57.134262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b0e49-3cf7-4d46-afd1-728e5af6c4f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:57.144886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:57.145860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:57.146476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:57.146773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] PATH_INFO: `/v3/projects/6510317109e4468fb3d8c76b373b4e58/users/08da8aa73bcc47519fbe92233033e498/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:57.187458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6510317109e4468fb3d8c76b373b4e58, user_id=08da8aa73bcc47519fbe92233033e498, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:57.194743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:57.225357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 962/1885] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:57 2022] PUT /identity/v3/projects/6510317109e4468fb3d8c76b373b4e58/users/08da8aa73bcc47519fbe92233033e498/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:57.227399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:57.228293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:57.228613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:57.286658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] MFA Rules not processed for user `08da8aa73bcc47519fbe92233033e498`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:57.288793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:57.369554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:57.370973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:57.473996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1886] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:57.480433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:57.622602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086615fe-ad49-444d-be4d-e3b4ee7d9617 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:57.644100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:57.644786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:57.645079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:57.645364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] PATH_INFO: `/v3/users/1048371bc1c64f15b87e1462b16fdd3f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:57.646655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1048371bc1c64f15b87e1462b16fdd3f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:57.650109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:57.735766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7eba6-87b4-4dc1-acc1-9b6d0631fdf5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:57.743699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1048371bc1c64f15b87e1462b16fdd3f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:57.748984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:57.750014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:57.750527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:57.751029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:57.751978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:57.754402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:57.755977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1048371bc1c64f15b87e1462b16fdd3f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:57.757868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1048371bc1c64f15b87e1462b16fdd3f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:57.759552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1048371bc1c64f15b87e1462b16fdd3f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:57.785120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4372860-437b-4c07-ae54-512c57e8ec1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1048371bc1c64f15b87e1462b16fdd3f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:57.828396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1887] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:57 2022] POST /identity/v3/projects => generated 393 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:57.832402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:57.834144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:57.844376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 963/1888] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:57 2022] DELETE /identity/v3/users/1048371bc1c64f15b87e1462b16fdd3f => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:57.848316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:57.849405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:57.851585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:57.904682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:57.906881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:57.932167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:57.949952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610ce3b-f354-4cf9-a65a-049820d3a03e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:57.969829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:58.037310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 964/1889] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:21:58.046352 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.047001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.047331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.047635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.048655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.049695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.054587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.054825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.088511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1890] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:21:57 2022] POST /identity/v3/users => generated 303 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:21:58.093140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.108064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f8f6-db61-44ac-825e-eb2da098d74b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.113979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.114627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.114918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.115200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.115980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.118291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.131955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1891] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:58.135874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.136999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f9ba-9755-4a3f-98a2-81275ebf60c9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.141071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.141793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.142203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.143034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.147367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54bd4f-78ae-4a48-813f-935be1034ce9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.153102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.153982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.154390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.154575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] PATH_INFO: `/v3/projects/8d3f5717bb52462f97659a2a92aa16a7/users/b2bb27cb93894ccba2a3afb4f5b9c477/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.155116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.156655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.170886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 965/1892] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:58.189218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3f5717bb52462f97659a2a92aa16a7, user_id=b2bb27cb93894ccba2a3afb4f5b9c477, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.192696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.216122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1893] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:58 2022] PUT /identity/v3/projects/8d3f5717bb52462f97659a2a92aa16a7/users/b2bb27cb93894ccba2a3afb4f5b9c477/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:58.219865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.231776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f1081-6f12-4bbf-b06c-a2ffa6027f4f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.312362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b68bc-e26a-4a6b-a8ff-4d92b1082b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.313579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b68bc-e26a-4a6b-a8ff-4d92b1082b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.314179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b68bc-e26a-4a6b-a8ff-4d92b1082b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.314499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b68bc-e26a-4a6b-a8ff-4d92b1082b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.316306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b68bc-e26a-4a6b-a8ff-4d92b1082b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.318118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b68bc-e26a-4a6b-a8ff-4d92b1082b31 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.334092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 966/1894] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:21:58.338125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.389033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a7f32-6de1-425e-842f-de579bf14216 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.401929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.402733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.403023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.403643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] PATH_INFO: `/v3/projects/8d3f5717bb52462f97659a2a92aa16a7/users/b2bb27cb93894ccba2a3afb4f5b9c477/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.409309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3f5717bb52462f97659a2a92aa16a7, user_id=b2bb27cb93894ccba2a3afb4f5b9c477, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.412480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.425677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1895] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:21:58 2022] PUT /identity/v3/projects/8d3f5717bb52462f97659a2a92aa16a7/users/b2bb27cb93894ccba2a3afb4f5b9c477/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:58.436179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.436368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.436660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.468271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] MFA Rules not processed for user `b2bb27cb93894ccba2a3afb4f5b9c477`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:58.470129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.544254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:58.574137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 967/1896] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:58.584353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.586596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.618708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.620241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.682111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d6bd1-14e7-4e6d-9f0f-445d4bc1f9dc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.687599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.689899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.690208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.690997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.702934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.704556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.735835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ce12-e4b8-42ae-93b2-ae1b487c83eb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.741970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.742436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1897] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:21:58.742724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.743017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.743297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] PATH_INFO: `/v3/projects/5c5f38cd664246748d2a93e4133a928b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.744532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5f38cd664246748d2a93e4133a928b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:58.746479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:58.808261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.810099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.829404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629fa9d-fef2-4192-87c5-9052d2dd7bc1 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:58.833050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5f38cd664246748d2a93e4133a928b', 'request_id': 'req-c21e0700-5dc8-416e-bad8-41e4731c8d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:58.847960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5f38cd664246748d2a93e4133a928b', 'request_id': 'req-c21e0700-5dc8-416e-bad8-41e4731c8d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:58.875021 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5f38cd664246748d2a93e4133a928b', 'request_id': 'req-c21e0700-5dc8-416e-bad8-41e4731c8d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:58.903004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] The token cache is being invalidate because project 5c5f38cd664246748d2a93e4133a928b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:58.904528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:58.904733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:58.922701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 968/1898] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:58 2022] DELETE /identity/v3/projects/5c5f38cd664246748d2a93e4133a928b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:58.927945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:58.942447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:58.943464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:58.943630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.943841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:58.944157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:58.946401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:58.963395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:59.028469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:59.030234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:59.042438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1899] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:21:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:21:59.067029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e0700-5dc8-416e-bad8-41e4731c8d91 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:59.073370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:59.074457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:59.074926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:59.075422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] PATH_INFO: `/v3/users/1979a6aa5d0041418d22060c87886d06` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:59.076914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1979a6aa5d0041418d22060c87886d06)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:59.079171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:59.108191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:21:59.151498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1979a6aa5d0041418d22060c87886d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.172279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1979a6aa5d0041418d22060c87886d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.173578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1979a6aa5d0041418d22060c87886d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.177253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1979a6aa5d0041418d22060c87886d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.194756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b11d2052-ca26-48cf-904c-4b3bfbca81fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1979a6aa5d0041418d22060c87886d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.209578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 969/1900] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:21:58 2022] DELETE /identity/v3/users/1979a6aa5d0041418d22060c87886d06 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:21:59.581219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:59.599173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43625c-9e79-4abd-b55b-3a6d9b86e4fd service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:59.680236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:59.680907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:59.681195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:59.681474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] PATH_INFO: `/v3/projects/3bf975a185064fbeaba8486ada433e1c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:59.682668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf975a185064fbeaba8486ada433e1c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:59.684785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:59.741006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf975a185064fbeaba8486ada433e1c', 'request_id': 'req-0914d9b2-a2a3-4080-869a-d4f25fd8498a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.747542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf975a185064fbeaba8486ada433e1c', 'request_id': 'req-0914d9b2-a2a3-4080-869a-d4f25fd8498a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.758378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf975a185064fbeaba8486ada433e1c', 'request_id': 'req-0914d9b2-a2a3-4080-869a-d4f25fd8498a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.773505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] The token cache is being invalidate because project 3bf975a185064fbeaba8486ada433e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:21:59.773925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.774586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0914d9b2-a2a3-4080-869a-d4f25fd8498a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:21:59.776360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1901] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:21:59 2022] DELETE /identity/v3/projects/3bf975a185064fbeaba8486ada433e1c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:21:59.781182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:59.781652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:59.781744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:59.804179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:59.805655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:59.878755 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 970/1902] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:59.886903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:59.887194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:59.887523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:59.911793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:21:59.913509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:21:59.927625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1903] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:21:59.928350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:59.940714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20978041-9141-44d3-9293-ceb6a4064f35 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:59.945963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:59.946615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:59.946905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:21:59.947187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:21:59.947984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:21:59.948299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:21:59.949966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:21:59.965554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 971/1904] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:21:59.969817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:21:59.981245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cb829-b2d6-4ecc-a4ff-bb43ecc53e5c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:21:59.998586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:21:59.999423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:21:59.999818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.000164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.000920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.003335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.041970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1905] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:21:59 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:00.045769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.056363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc586ef-2480-418d-b507-994dc4896ee6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.135012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.135676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.135960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.136232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.137245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.139119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.162209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 972/1906] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:00 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:00.166670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.178390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dda31-7d4f-45fa-bd11-045e20ab9312 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.183314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.183978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.184275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.185078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.185409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.187048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.200627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1907] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:00.204617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.215075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad60f1-591b-4468-9970-d6f79279a60b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.221230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c28873-1676-4c8a-be2d-84c4789fb3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.221926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c28873-1676-4c8a-be2d-84c4789fb3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.222209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c28873-1676-4c8a-be2d-84c4789fb3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.222616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c28873-1676-4c8a-be2d-84c4789fb3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.223710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c28873-1676-4c8a-be2d-84c4789fb3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.226107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c28873-1676-4c8a-be2d-84c4789fb3bb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.239499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 973/1908] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:00.242740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.250411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:00.256008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5bea-89d9-4048-8210-79f52d765cc0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.263598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.264488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.264866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.265084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] PATH_INFO: `/v3/projects/d5cf946743a24578a048fc305a6e1882/users/22480601e35a4153b421667dcf7dbc95/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.280970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5cf946743a24578a048fc305a6e1882, user_id=22480601e35a4153b421667dcf7dbc95, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.283763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.297098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1909] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:00 2022] PUT /identity/v3/projects/d5cf946743a24578a048fc305a6e1882/users/22480601e35a4153b421667dcf7dbc95/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:00.309318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.309639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.309890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.319135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:00.342987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] MFA Rules not processed for user `22480601e35a4153b421667dcf7dbc95`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:00.344303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:00.416101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 974/1910] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:00.420586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.497843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd6788-025b-4aa6-94f7-1c52d2298204 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.505154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.505790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.506074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.506357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.507083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.509290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.541710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1911] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:00 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:00.543095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.555798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5397cc8-45bc-424e-b092-f474b78a141a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.616759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.617387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.617670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.617952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.618629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.620602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.642676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 975/1912] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:00 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:00.646893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.657688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbd638-c50f-4f13-90ad-4cada280c0a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.663913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.664561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.664849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.665131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.665838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.667517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.679532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1913] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:00.683914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.694847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f082b-e687-415f-8a0d-df3b78ab5fcd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.699874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.700567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.700873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.701139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] PATH_INFO: `/v3/projects/b81eb83d09984766bee37278db1572aa/users/1a904fad7d204f4099ded638f4d2657d/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.718827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81eb83d09984766bee37278db1572aa, user_id=1a904fad7d204f4099ded638f4d2657d, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.722535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.735888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 976/1914] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:00 2022] PUT /identity/v3/projects/b81eb83d09984766bee37278db1572aa/users/1a904fad7d204f4099ded638f4d2657d/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:00.741330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.751756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfc513-9ae3-45ec-92bb-fd2952b23967 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.822932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab76c9-1dab-42db-b803-6173a6811d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.824207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab76c9-1dab-42db-b803-6173a6811d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.824500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab76c9-1dab-42db-b803-6173a6811d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.824942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab76c9-1dab-42db-b803-6173a6811d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.825812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab76c9-1dab-42db-b803-6173a6811d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.829589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab76c9-1dab-42db-b803-6173a6811d68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.840244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1915] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:00.844388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:00.856171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a4ba-1ac1-4f4f-9739-4b344bb41950 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:00.861119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:00.862755 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.863081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.863497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] PATH_INFO: `/v3/projects/b81eb83d09984766bee37278db1572aa/users/1a904fad7d204f4099ded638f4d2657d/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.865821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81eb83d09984766bee37278db1572aa, user_id=1a904fad7d204f4099ded638f4d2657d, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:00.868576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:00.885295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 977/1916] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:00 2022] PUT /identity/v3/projects/b81eb83d09984766bee37278db1572aa/users/1a904fad7d204f4099ded638f4d2657d/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:00.896715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:00.897118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:00.897360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:00.929831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] MFA Rules not processed for user `1a904fad7d204f4099ded638f4d2657d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:00.931504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:00.996777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:01.043316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1917] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:01.055144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:01.056942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:01.144128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:01.151654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20612f-df07-47af-81f1-09611be6d9aa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:01.155948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f94bb8-8a8f-418b-95d9-1538fde05c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:01.156697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f94bb8-8a8f-418b-95d9-1538fde05c13 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:01.157045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f94bb8-8a8f-418b-95d9-1538fde05c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:01.157400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f94bb8-8a8f-418b-95d9-1538fde05c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:01.233275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f94bb8-8a8f-418b-95d9-1538fde05c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:01.235147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f94bb8-8a8f-418b-95d9-1538fde05c13 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:01.246724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 978/1918] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:03.885330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:03.886851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:03.971597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f21ffd-9f4d-40d4-8fc9-3c4ac6f4cdc4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:03.990858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:03.992073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:03.993199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:03.993556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] PATH_INFO: `/v3/users/08da8aa73bcc47519fbe92233033e498` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:03.994941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=08da8aa73bcc47519fbe92233033e498)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:03.997737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:04.102141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08da8aa73bcc47519fbe92233033e498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:04.103242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08da8aa73bcc47519fbe92233033e498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:04.123904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08da8aa73bcc47519fbe92233033e498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:04.125215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08da8aa73bcc47519fbe92233033e498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:04.156082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08da8aa73bcc47519fbe92233033e498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:04.197321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1919] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:03 2022] DELETE /identity/v3/users/08da8aa73bcc47519fbe92233033e498 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:04.202308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:04.202732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:04.202957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:04.249958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:04.257142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:04.391650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 979/1920] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:04.409774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:04.411296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:04.505649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73fd24-2d6e-4dcc-8d15-1db8bea84c8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:04.520721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:04.521994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:04.522522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:04.523184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:04.545968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:04.550994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:04.563984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:04.583180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1921] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:05.084412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:05.086490 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:05.100284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb494e5-3fd4-4f38-9809-ee754208b63e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:05.105534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:05.106222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:05.106514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:05.106815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] PATH_INFO: `/v3/projects/6510317109e4468fb3d8c76b373b4e58` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:05.108110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6510317109e4468fb3d8c76b373b4e58)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:05.110160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:05.166554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6510317109e4468fb3d8c76b373b4e58', 'request_id': 'req-3f155606-b0ca-44ee-8cef-18acabd6d345'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.178310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6510317109e4468fb3d8c76b373b4e58', 'request_id': 'req-3f155606-b0ca-44ee-8cef-18acabd6d345'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.192559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6510317109e4468fb3d8c76b373b4e58', 'request_id': 'req-3f155606-b0ca-44ee-8cef-18acabd6d345'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.214797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] The token cache is being invalidate because project 6510317109e4468fb3d8c76b373b4e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:05.215466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.215875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.217474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 980/1922] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:05 2022] DELETE /identity/v3/projects/6510317109e4468fb3d8c76b373b4e58 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:05.221763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:05.232796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f4ee4-fa63-4602-8a06-90612c5f3e25 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:05.297869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:05.299059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:05.299385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:05.299662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] PATH_INFO: `/v3/users/b2bb27cb93894ccba2a3afb4f5b9c477` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:05.300899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2bb27cb93894ccba2a3afb4f5b9c477)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:05.302893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:05.359640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2bb27cb93894ccba2a3afb4f5b9c477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.360338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2bb27cb93894ccba2a3afb4f5b9c477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.368901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2bb27cb93894ccba2a3afb4f5b9c477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.369584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2bb27cb93894ccba2a3afb4f5b9c477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.382778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa0bb53a-734f-47e5-8ce8-5205b495b342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2bb27cb93894ccba2a3afb4f5b9c477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.397254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1923] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:05 2022] DELETE /identity/v3/users/b2bb27cb93894ccba2a3afb4f5b9c477 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:05.777334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:05.779233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:05.806595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155606-b0ca-44ee-8cef-18acabd6d345 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:05.831882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:05.884565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:05.885222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:05.885508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:05.885787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] PATH_INFO: `/v3/projects/8d3f5717bb52462f97659a2a92aa16a7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:05.886946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3f5717bb52462f97659a2a92aa16a7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:05.888919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:05.942093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3f5717bb52462f97659a2a92aa16a7', 'request_id': 'req-6fb6220e-7283-47ea-bf71-41dcf0a52829'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.952554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3f5717bb52462f97659a2a92aa16a7', 'request_id': 'req-6fb6220e-7283-47ea-bf71-41dcf0a52829'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.962697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3f5717bb52462f97659a2a92aa16a7', 'request_id': 'req-6fb6220e-7283-47ea-bf71-41dcf0a52829'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.977727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] The token cache is being invalidate because project 8d3f5717bb52462f97659a2a92aa16a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:05.978324 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.979184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6fb6220e-7283-47ea-bf71-41dcf0a52829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:05.980954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 981/1924] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:05 2022] DELETE /identity/v3/projects/8d3f5717bb52462f97659a2a92aa16a7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:05.986830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:05.987190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:05.987452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.009203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:06.010554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:06.095133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1925] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:06.105929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.106242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.106530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.127642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:06.128972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:06.145916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 982/1926] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:06.150480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.162880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9abcf3-3405-4a84-a96c-ea648de6390d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.167598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.168242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.168530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.168814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.169531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:06.169869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.171648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.184166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1927] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:06.188709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.199324 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a0863-44b2-4bc1-9f67-55aed8d26685 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.204421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.206457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.206745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.207022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.207747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.209711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.244507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 983/1928] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:06 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:06.246010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.252855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:06.257560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836b718-4274-4f0b-8c1f-309ba49e8f95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.320471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.321109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.321398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.321707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.322383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.324363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.347895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1929] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:06 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:06.351726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.363012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d4345-152f-464e-a6b1-7e67b2bfffaf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.368055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.368781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.368984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.369264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.369972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.371616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.383943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 984/1930] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:06.388024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.399025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4fc54-428c-4870-a723-20675378eeac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.403836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17815c-859c-46fb-b981-987feffb80ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.404466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17815c-859c-46fb-b981-987feffb80ba admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.404757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17815c-859c-46fb-b981-987feffb80ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.405046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17815c-859c-46fb-b981-987feffb80ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.405740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17815c-859c-46fb-b981-987feffb80ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.407375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17815c-859c-46fb-b981-987feffb80ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.418996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1931] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:06.423337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.434612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5cba1-3d3f-40e8-9454-89e7f8344d4c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.439772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.440460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.440751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.441036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] PATH_INFO: `/v3/projects/73738f4f53df44cfb474d8276dc80919/users/bfce973a4676466589a8cf5a76db948c/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.456187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73738f4f53df44cfb474d8276dc80919, user_id=bfce973a4676466589a8cf5a76db948c, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.458851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.473180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 985/1932] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:06 2022] PUT /identity/v3/projects/73738f4f53df44cfb474d8276dc80919/users/bfce973a4676466589a8cf5a76db948c/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:06.484726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.485018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.485297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.514601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] MFA Rules not processed for user `bfce973a4676466589a8cf5a76db948c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:06.515868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:06.583197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1933] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:06.587764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.639675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:06.652132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd617b8-57f8-41a7-bf21-cd11461b5a38 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.657251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.658423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.658701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.658981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.659852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.661635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.693562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 986/1934] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:06 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:06.696453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.708110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2de5c4-bc09-4d1c-a155-a8f9fb056f86 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.772858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.773510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.773788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.774078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.775243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.776743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.801474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1935] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:06 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:06.805669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.818357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad62d20-172c-46df-b40c-3c19d7053856 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.825293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.826207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.826497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.826774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.827516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.829227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.842645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 987/1936] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:06.847550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.853064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:06.861469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e33c1-e4c7-4efd-97f3-0e4631cfb58a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.867166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.867971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.869825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.870243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] PATH_INFO: `/v3/projects/904f61bc3a854d5a85e0446e2313cdc2/users/c3c829b9ebdd462ba73e3257b9959e5b/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.890417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904f61bc3a854d5a85e0446e2313cdc2, user_id=c3c829b9ebdd462ba73e3257b9959e5b, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.894018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:06.913700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1937] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:06 2022] PUT /identity/v3/projects/904f61bc3a854d5a85e0446e2313cdc2/users/c3c829b9ebdd462ba73e3257b9959e5b/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:06.918144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:06.928252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94cf5-dce8-4c7d-9a3a-a30744675a87 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:06.990489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d73517-5a3d-4c17-a236-fa9136201f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:06.991245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73517-5a3d-4c17-a236-fa9136201f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:06.991556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73517-5a3d-4c17-a236-fa9136201f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:06.991834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73517-5a3d-4c17-a236-fa9136201f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:06.992529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d73517-5a3d-4c17-a236-fa9136201f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:06.994137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d73517-5a3d-4c17-a236-fa9136201f81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:07.004777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 988/1938] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:07.008871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:07.021058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ba10a-18c7-437d-84fa-7af5397e64a8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:07.026248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:07.027067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:07.027448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:07.028072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] PATH_INFO: `/v3/projects/904f61bc3a854d5a85e0446e2313cdc2/users/c3c829b9ebdd462ba73e3257b9959e5b/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:07.030119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904f61bc3a854d5a85e0446e2313cdc2, user_id=c3c829b9ebdd462ba73e3257b9959e5b, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:07.033643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:07.053875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1939] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:07 2022] PUT /identity/v3/projects/904f61bc3a854d5a85e0446e2313cdc2/users/c3c829b9ebdd462ba73e3257b9959e5b/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:07.064225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:07.064523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:07.064815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:07.097153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] MFA Rules not processed for user `c3c829b9ebdd462ba73e3257b9959e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:07.098630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:07.194742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 989/1940] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:07.207624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:07.209643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:07.277229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa115576-e697-4654-9c24-4b4dde1d6cd3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:07.281070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:07.281799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:07.282159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:07.282632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:07.295104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:07.296899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:07.312647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1941] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:08.005592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:08.076363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:08.097998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e9174-5ebf-436b-8e24-31332f0dc447 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:08.100407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:08.101914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:08.117208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:08.118119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:08.118415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:08.118697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] PATH_INFO: `/v3/users/bfce973a4676466589a8cf5a76db948c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:08.122968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfce973a4676466589a8cf5a76db948c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:08.123958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46910e-3f57-4878-88b3-fee7617a8b69 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:08.126209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:08.132453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:08.133103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:08.133394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:08.133675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:08.138565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:08.203762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfce973a4676466589a8cf5a76db948c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:08.214375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfce973a4676466589a8cf5a76db948c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:08.215546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfce973a4676466589a8cf5a76db948c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:08.216118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfce973a4676466589a8cf5a76db948c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:08.233099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a9ff0e19-902e-42f6-913f-32a8154d6f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfce973a4676466589a8cf5a76db948c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:08.242671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:08.244196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:08.251497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 990/1942] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:08 2022] DELETE /identity/v3/users/bfce973a4676466589a8cf5a76db948c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:08.255387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1943] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:08.256347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:08.256872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:08.257161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:08.279954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:08.281267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:08.380556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 991/1944] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:08.390022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:08.393208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:08.528427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895790-36ca-41de-a66b-4c208139a3d9 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:08.546931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:08.605374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:08.606814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:08.607161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:08.607453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:08.620865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:08.622571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:08.638150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1945] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:09.910002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:09.912287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:09.937849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efb741-66f2-4dc7-8cba-c5bdf5b7c424 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:09.948345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:09.950893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:09.952195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:09.953231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] PATH_INFO: `/v3/projects/73738f4f53df44cfb474d8276dc80919` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:09.957517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=73738f4f53df44cfb474d8276dc80919)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:09.961280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:10.034056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73738f4f53df44cfb474d8276dc80919', 'request_id': 'req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.045625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73738f4f53df44cfb474d8276dc80919', 'request_id': 'req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.054849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:10.057087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73738f4f53df44cfb474d8276dc80919', 'request_id': 'req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.078005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] The token cache is being invalidate because project 73738f4f53df44cfb474d8276dc80919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:10.078686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.080201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.085518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 992/1946] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:09 2022] DELETE /identity/v3/projects/73738f4f53df44cfb474d8276dc80919 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:10.087246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:10.111238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a0d47-274c-49ea-a97f-cb9e601f11a7 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:10.190810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:10.192201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:10.192201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:10.192201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] PATH_INFO: `/v3/users/c3c829b9ebdd462ba73e3257b9959e5b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:10.193530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c829b9ebdd462ba73e3257b9959e5b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:10.200065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:10.276593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c829b9ebdd462ba73e3257b9959e5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.277342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c829b9ebdd462ba73e3257b9959e5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.288152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c829b9ebdd462ba73e3257b9959e5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.289357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c829b9ebdd462ba73e3257b9959e5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.307172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a232e0f8-c688-4479-bc79-dff428c08a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c829b9ebdd462ba73e3257b9959e5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:10.346766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1947] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:10 2022] DELETE /identity/v3/users/c3c829b9ebdd462ba73e3257b9959e5b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:11.228651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:11.231433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:11.261417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4c344-ab8b-4ad7-b35d-a93c6c3f0d5a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:11.350572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:11.351641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:11.351955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:11.352238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] PATH_INFO: `/v3/projects/904f61bc3a854d5a85e0446e2313cdc2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:11.353746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=904f61bc3a854d5a85e0446e2313cdc2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:11.355778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:11.417536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904f61bc3a854d5a85e0446e2313cdc2', 'request_id': 'req-63c23b59-e4bb-45eb-b530-039a65138ea3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:11.440395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904f61bc3a854d5a85e0446e2313cdc2', 'request_id': 'req-63c23b59-e4bb-45eb-b530-039a65138ea3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:11.467871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904f61bc3a854d5a85e0446e2313cdc2', 'request_id': 'req-63c23b59-e4bb-45eb-b530-039a65138ea3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:11.513060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] The token cache is being invalidate because project 904f61bc3a854d5a85e0446e2313cdc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:11.515365 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:11.519108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-63c23b59-e4bb-45eb-b530-039a65138ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:11.525883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 993/1948] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:11 2022] DELETE /identity/v3/projects/904f61bc3a854d5a85e0446e2313cdc2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:11.536554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:11.544015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:11.545117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:11.644900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:11.649527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:11.731461 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:11.793500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:11.793963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1949] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:11.794773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:11.794773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:11.837192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:11.840573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:11.871197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 994/1950] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:11.875245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:11.895934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369773c0-5ed6-4fbb-ae57-376e33318d68 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:11.906332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:11.908365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:11.909076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:11.909997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:11.912754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:11.913489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:11.917742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:11.945087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1951] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:11.951179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:11.971302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959a59f-3a6f-44e1-b75d-a2508bfc848c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:11.979476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:11.980578 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:11.981092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:11.981667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:11.983059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:11.987604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:12.072984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 995/1952] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:11 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:12.077554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:12.096285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833b992-d2fd-4dc3-9418-b12e34cccf6a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:12.351588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:12.352257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:12.353430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:12.353430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:12.355226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:12.357410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:12.388954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1953] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:12 2022] POST /identity/v3/users => generated 291 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:12.394287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:12.409306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4022f-29c7-4e06-9962-4f2a995316ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:12.426969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:12.428089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:12.428564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:12.429447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:12.430491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:12.432242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:12.457646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 996/1954] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:12.462330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:12.474711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd10cda-bc1c-4f17-87dd-e999c1edf1c5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:12.479934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b564d7-ebd3-4d91-8556-10481bcda56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:12.481206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b564d7-ebd3-4d91-8556-10481bcda56e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:12.481206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b564d7-ebd3-4d91-8556-10481bcda56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:12.481206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b564d7-ebd3-4d91-8556-10481bcda56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:12.481871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b564d7-ebd3-4d91-8556-10481bcda56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:12.483582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b564d7-ebd3-4d91-8556-10481bcda56e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:12.497838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1955] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:12.501482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:12.513788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f532e5-5e84-4db4-8da2-82ed4f6aa00b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:12.520117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:12.521157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:12.521612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:12.522106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] PATH_INFO: `/v3/projects/a5090b4e2de34d2dba18758140b5826b/users/edf86acfe5704e71baa368675aad7a84/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:12.547723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5090b4e2de34d2dba18758140b5826b, user_id=edf86acfe5704e71baa368675aad7a84, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:12.553465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:12.599676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 997/1956] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:12 2022] PUT /identity/v3/projects/a5090b4e2de34d2dba18758140b5826b/users/edf86acfe5704e71baa368675aad7a84/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:12.612840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:12.613755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:12.614226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:12.653863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] MFA Rules not processed for user `edf86acfe5704e71baa368675aad7a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:12.655287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:12.757983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1957] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:12.763659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:12.792780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:12.860415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906f68c-0037-4a88-a86a-ef601aa489e5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:12.866552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:12.866838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:12.867179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:12.867480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:12.868201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:12.870626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:12.908928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 998/1958] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:12 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:12.910859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:12.927197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22b7e-999d-4d12-bbaa-8f3423014bf8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:12.973648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:13.016103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:13.017427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.017833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.018741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.021304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:13.024056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:13.058841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1959] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:12 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:13.062683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:13.084606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35193a9-c0c1-44f2-a468-256a84f2f66b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:13.091039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:13.091319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.091645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.091895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.092690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:13.096316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:13.124260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 999/1960] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:13.129455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:13.165958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d307e8-bd26-45dc-a11b-afe7aee71156 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:13.177255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:13.177976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.178261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.178543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] PATH_INFO: `/v3/projects/8dbc53f197bc462d95d93e282b761e80/users/20f69db40a4f48b3b215912b273b96a3/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.234595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbc53f197bc462d95d93e282b761e80, user_id=20f69db40a4f48b3b215912b273b96a3, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:13.242850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:13.266131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1961] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:13 2022] PUT /identity/v3/projects/8dbc53f197bc462d95d93e282b761e80/users/20f69db40a4f48b3b215912b273b96a3/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:13.270532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:13.284681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bde17d-322a-43c2-b7f0-89b48981d279 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:13.409131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:13.411637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.412219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.412806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.413639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:13.419444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:13.443641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1000/1962] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:13.446306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:13.459972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42de94-596f-4106-b17c-db1a836972e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:13.466467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51b422-45ae-422d-af7d-635c2b5ca37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:13.467278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51b422-45ae-422d-af7d-635c2b5ca37d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.467995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51b422-45ae-422d-af7d-635c2b5ca37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.468976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51b422-45ae-422d-af7d-635c2b5ca37d admin admin] PATH_INFO: `/v3/projects/8dbc53f197bc462d95d93e282b761e80/users/20f69db40a4f48b3b215912b273b96a3/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.472100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51b422-45ae-422d-af7d-635c2b5ca37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbc53f197bc462d95d93e282b761e80, user_id=20f69db40a4f48b3b215912b273b96a3, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:13.475022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51b422-45ae-422d-af7d-635c2b5ca37d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:13.503270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1963] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:13 2022] PUT /identity/v3/projects/8dbc53f197bc462d95d93e282b761e80/users/20f69db40a4f48b3b215912b273b96a3/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:13.512272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:13.513422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.513712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.513984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.514977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:13.530725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7838651-2ae2-49ab-9703-db0f518669f8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:13.546898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] MFA Rules not processed for user `20f69db40a4f48b3b215912b273b96a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:13.548738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:13.612369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:13.613313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.614011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.614405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] PATH_INFO: `/v3/users/76273fd3588d45be816d2c57636f3226` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.615788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76273fd3588d45be816d2c57636f3226)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:13.617827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:13.678208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:13.724915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76273fd3588d45be816d2c57636f3226'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:13.747188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76273fd3588d45be816d2c57636f3226'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:13.747989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76273fd3588d45be816d2c57636f3226'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:13.748789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76273fd3588d45be816d2c57636f3226'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:13.761979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1964] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:13.777123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6b8745c-34ee-43a5-8f90-c170c207ba4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76273fd3588d45be816d2c57636f3226'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:13.822025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:13.827990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1001/1965] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:13 2022] DELETE /identity/v3/users/76273fd3588d45be816d2c57636f3226 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:13.828247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:13.830746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:13.831104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:13.831377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:13.879422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:13.881023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:13.988038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49182559-0d36-45cd-b35d-d3c34b5a4bdc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:14.043624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1002/1966] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:14.065712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:14.075869 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:14.080312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:14.143584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:14.145294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:14.145467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:14.145907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:14.149233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:14.189307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88254835-6ee4-4654-ad83-dc7186c2eeba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:14.196116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:14.197539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:14.198047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:14.198586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:14.237481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:14.240272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:14.274692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1003/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:14.288596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:14.290581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:14.327858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bcad6-3e35-4d69-a57f-874cb7d215c6 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:14.342085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:14.343187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:14.343729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:14.344261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] PATH_INFO: `/v3/users/22480601e35a4153b421667dcf7dbc95` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:14.346657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=22480601e35a4153b421667dcf7dbc95)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:14.349217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:14.388661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:14.394544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c88564-120b-4643-b00b-2b36de235462 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:14.444864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1968] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:14.468229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:14.476711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22480601e35a4153b421667dcf7dbc95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:14.503484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22480601e35a4153b421667dcf7dbc95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:14.505268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22480601e35a4153b421667dcf7dbc95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:14.507413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22480601e35a4153b421667dcf7dbc95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:14.547579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22480601e35a4153b421667dcf7dbc95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:14.597670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1004/1969] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:14 2022] DELETE /identity/v3/users/22480601e35a4153b421667dcf7dbc95 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:14.602174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:14.602682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:14.603651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:14.658843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:14.660181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:14.799548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1970] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:14.813258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:14.816235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:14.925819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2540bca-ab5d-42d8-9f5e-0ab382ae19cd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:14.930117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:14.931095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:14.931698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:14.932126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:14.951673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:14.953418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:14.981970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1005/1971] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:15.044767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:15.046690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:15.135367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d66dc-c1d0-45de-a4ff-8bb3e2a82cb6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:15.148304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:15.149041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.149334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.149617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] PATH_INFO: `/v3/projects/02466603267e4f8b8bbc7ca347b09124` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.150994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=02466603267e4f8b8bbc7ca347b09124)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:15.154337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:15.217932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:15.219453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:15.244344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d30a-7f14-49b6-b0c0-c96805f6f06b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:15.268923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02466603267e4f8b8bbc7ca347b09124', 'request_id': 'req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.286448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02466603267e4f8b8bbc7ca347b09124', 'request_id': 'req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.299318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02466603267e4f8b8bbc7ca347b09124', 'request_id': 'req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.333550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] The token cache is being invalidate because project 02466603267e4f8b8bbc7ca347b09124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:15.334580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.336158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8d2d9ae-c69d-4f8b-a854-8075bda094ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.339793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1972] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:15 2022] DELETE /identity/v3/projects/02466603267e4f8b8bbc7ca347b09124 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:15.346541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9e908-ed8e-4702-a969-7903de3b776a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.346999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9e908-ed8e-4702-a969-7903de3b776a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.347407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9e908-ed8e-4702-a969-7903de3b776a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.357409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:15.383495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28e9e908-ed8e-4702-a969-7903de3b776a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:15.385910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28e9e908-ed8e-4702-a969-7903de3b776a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:15.411063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:15.413964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.414487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.414978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] PATH_INFO: `/v3/users/edf86acfe5704e71baa368675aad7a84` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.431420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf86acfe5704e71baa368675aad7a84)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:15.440800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:15.498153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1973] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:15.508608 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.508996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.509340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.578761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf86acfe5704e71baa368675aad7a84'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.580515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:15.582650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:15.593433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf86acfe5704e71baa368675aad7a84'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.594590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf86acfe5704e71baa368675aad7a84'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.596124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf86acfe5704e71baa368675aad7a84'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.604824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1974] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:15.609656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:15.616321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:15.628629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270085e7-43ac-4414-ac67-bf1b8d6e8ada None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:15.635907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bfa4e949-64b2-4ebd-aae4-05749cbf6c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf86acfe5704e71baa368675aad7a84'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:15.649116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:15.649783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.650072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.650355 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.651772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:15.651772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:15.653272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:15.684071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1006/1975] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:15 2022] DELETE /identity/v3/users/edf86acfe5704e71baa368675aad7a84 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:15.703947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.705102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.705256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.716165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1976] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:22:15.725264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:15.728113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:15.748084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:15.752393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:15.766128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d9e47-b1f9-44ab-a0c1-ee4ecec500e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:15.920474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:15.921779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:15.922075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:15.922363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:15.926520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1007/1977] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:15.933858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:15.937751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:15.941257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:15.943335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:16.004240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1978] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:15 2022] POST /identity/v3/projects => generated 383 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:16.005495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:16.007810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:16.065517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6686a0-130b-49cc-b9af-198559a47a8d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.104854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab558bc-aba0-421b-b370-ac38a09800bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.113642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.114339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.114626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.114907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] PATH_INFO: `/v3/projects/d5cf946743a24578a048fc305a6e1882` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.116420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5cf946743a24578a048fc305a6e1882)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.118309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.131947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:16.191936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.193008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.193214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.193765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.210861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5cf946743a24578a048fc305a6e1882', 'request_id': 'req-d6049229-9b04-4a20-b2b8-da51671b5eb3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.221594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.222284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5cf946743a24578a048fc305a6e1882', 'request_id': 'req-d6049229-9b04-4a20-b2b8-da51671b5eb3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.225414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.231338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5cf946743a24578a048fc305a6e1882', 'request_id': 'req-d6049229-9b04-4a20-b2b8-da51671b5eb3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.241017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1008/1979] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:16.242056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:16.254146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa704b-a227-4cad-a8ba-68270291f0cd service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.273437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] The token cache is being invalidate because project d5cf946743a24578a048fc305a6e1882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:16.273987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.276312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.285634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1980] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:16 2022] DELETE /identity/v3/projects/d5cf946743a24578a048fc305a6e1882 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:16.286870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:16.299667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:16.373081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:16.411837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6049229-9b04-4a20-b2b8-da51671b5eb3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.426086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.428408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.428695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.429332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.430087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.430523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.431597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.432257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.432749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] PATH_INFO: `/v3/users/1a904fad7d204f4099ded638f4d2657d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.436376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a904fad7d204f4099ded638f4d2657d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.437379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.443017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.504006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1009/1981] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:16 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:16.508477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:16.510960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:16.534609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca00704-2ec2-4690-8025-67743ac45bde admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.542555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.543905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.544877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.545216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.546623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.549526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.562431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a904fad7d204f4099ded638f4d2657d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.566030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a904fad7d204f4099ded638f4d2657d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.566321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1010/1982] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:16.569995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:16.586217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cf835-a89b-47d0-b266-6fc6d9697005 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.591948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.592694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.593057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.593408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.594257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.596428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.603466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a904fad7d204f4099ded638f4d2657d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.604735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a904fad7d204f4099ded638f4d2657d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.621215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1011/1983] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:16.623632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:16.627895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:16.651328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1272234-8d07-4a4c-a4e7-54244ee5000d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:16.659392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:16.660374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.660841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.661273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] PATH_INFO: `/v3/projects/c8e9dbbd088b43d68a5083138766251b/users/08b315584bc84bc188108633d3f3d5f3/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.662832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a229b95f-5aa3-441d-ba64-efd43765f1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a904fad7d204f4099ded638f4d2657d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:16.685236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e9dbbd088b43d68a5083138766251b, user_id=08b315584bc84bc188108633d3f3d5f3, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:16.689101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:16.700845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1984] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:16 2022] DELETE /identity/v3/users/1a904fad7d204f4099ded638f4d2657d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:16.725174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1012/1985] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:16 2022] PUT /identity/v3/projects/c8e9dbbd088b43d68a5083138766251b/users/08b315584bc84bc188108633d3f3d5f3/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:16.749940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:16.750833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:16.752290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:16.805740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] MFA Rules not processed for user `08b315584bc84bc188108633d3f3d5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:16.808613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:16.819880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:16.937913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1986] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:16.944069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.036508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.040685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:17.143745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa248b3-f284-42a7-8b12-3c8e073a666b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.153638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.154660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.155062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.155526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.158160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:17.160541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:17.163304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d566a67-3cc1-4944-bbcf-6eacd4f9a75f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.179720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.182868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.183393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.183593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] PATH_INFO: `/v3/projects/a5090b4e2de34d2dba18758140b5826b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.188073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5090b4e2de34d2dba18758140b5826b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:17.197244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:17.200842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1013/1987] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:16 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:17.208170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.349973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5090b4e2de34d2dba18758140b5826b', 'request_id': 'req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.368369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5090b4e2de34d2dba18758140b5826b', 'request_id': 'req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.385891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5090b4e2de34d2dba18758140b5826b', 'request_id': 'req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.412687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2a13-388d-42bc-9af5-a204406bed4d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.426493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] The token cache is being invalidate because project a5090b4e2de34d2dba18758140b5826b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:17.427071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.428458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.431834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1988] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:17 2022] DELETE /identity/v3/projects/a5090b4e2de34d2dba18758140b5826b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:17.438015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.440950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:17.453522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:17.519420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.520080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.520375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.520660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.521424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:17.523849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:17.544616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:17.554313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1014/1989] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:17 2022] POST /identity/v3/users => generated 293 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:17.555881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.559415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:17.574971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f84c-eb7e-4de0-bbc7-a1751083dfa3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.581652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.582654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.582948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.583461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] PATH_INFO: `/v3/projects/b81eb83d09984766bee37278db1572aa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.586981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81eb83d09984766bee37278db1572aa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:17.588402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2e23-3e7d-4091-8aec-12f1b7ec5f84 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.592090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:17.593675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.594695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.595773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.596353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] PATH_INFO: `/v3/users/20f69db40a4f48b3b215912b273b96a3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.597790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f69db40a4f48b3b215912b273b96a3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:17.600255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:17.670688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f69db40a4f48b3b215912b273b96a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.672018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f69db40a4f48b3b215912b273b96a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.676807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81eb83d09984766bee37278db1572aa', 'request_id': 'req-c976b644-1d7a-406e-969c-d08c471d6708'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.684970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81eb83d09984766bee37278db1572aa', 'request_id': 'req-c976b644-1d7a-406e-969c-d08c471d6708'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.686529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f69db40a4f48b3b215912b273b96a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.687761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f69db40a4f48b3b215912b273b96a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.694737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81eb83d09984766bee37278db1572aa', 'request_id': 'req-c976b644-1d7a-406e-969c-d08c471d6708'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.707455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3685a7d9-09bd-4865-8faf-237f0cebaf7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f69db40a4f48b3b215912b273b96a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.710687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] The token cache is being invalidate because project b81eb83d09984766bee37278db1572aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:17.711618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.711815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:17.713525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1015/1990] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:17 2022] DELETE /identity/v3/projects/b81eb83d09984766bee37278db1572aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:17.714479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.715956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:17.736769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1991] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:17 2022] DELETE /identity/v3/users/20f69db40a4f48b3b215912b273b96a3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:17.740014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06748667-1cae-445d-95c4-205b79eebb35 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.740014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06748667-1cae-445d-95c4-205b79eebb35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.740014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06748667-1cae-445d-95c4-205b79eebb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.771785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-06748667-1cae-445d-95c4-205b79eebb35 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:17.773057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06748667-1cae-445d-95c4-205b79eebb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:17.828773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976b644-1d7a-406e-969c-d08c471d6708 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.870975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1992] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:17.886389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.887976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.888381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.907719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:17.933308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:17.936499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:17.955378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1993] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:17.956362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.957071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.957449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:17.958001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:17.959327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:17.961326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.961523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:17.979733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f420-5041-48a6-82ef-b50390b52d87 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:17.987060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1016/1994] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:17.989502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:17.996934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:17.997920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:17.999568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.000056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.001702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:18.002024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.003894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.005722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738aeeb-30d4-4673-87f4-78665377acdb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.014489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.015342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.015767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.016136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.017891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.023627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1995] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:18.024572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.026766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.045612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1017/1996] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:18.047461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.047592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2a6e0-364b-4064-aaa8-b158a2b0810a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.057814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.059021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.059512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.060479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.060929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.067065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.068151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52693e-082a-45a5-b9a8-e275cfd19b86 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.078341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1852be-3948-43b5-b09f-2e3418b5f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.079669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1852be-3948-43b5-b09f-2e3418b5f129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.080335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1852be-3948-43b5-b09f-2e3418b5f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.081264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1852be-3948-43b5-b09f-2e3418b5f129 admin admin] PATH_INFO: `/v3/projects/a1fd58000bd549d2a45d6d6c61f1225e/users/b080f56558b2404ebb92ed210715674f/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.109378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1852be-3948-43b5-b09f-2e3418b5f129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fd58000bd549d2a45d6d6c61f1225e, user_id=b080f56558b2404ebb92ed210715674f, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.114316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1852be-3948-43b5-b09f-2e3418b5f129 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.160178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1997] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:18.161903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.176654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1018/1998] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:18 2022] PUT /identity/v3/projects/a1fd58000bd549d2a45d6d6c61f1225e/users/b080f56558b2404ebb92ed210715674f/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:18.201089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.201089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.201089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.222688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:18.258283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] MFA Rules not processed for user `b080f56558b2404ebb92ed210715674f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:18.260532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:18.327744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da746623-a337-48ae-89f8-d1430e2dbd4c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.431422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1019/1999] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:18.434881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.466164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.467283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.467834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.468325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.470070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.474353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.521132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/2000] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/users => generated 289 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:18.522883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.526889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:18.551086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d3b81-e705-4b8c-a96c-ecdf2bec7c18 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.557786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.557786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.557786 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.558362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] PATH_INFO: `/v3/projects/8dbc53f197bc462d95d93e282b761e80` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.560092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbc53f197bc462d95d93e282b761e80)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.562112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.578896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ebfb1-c084-411f-92cd-de8793ed364e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.587646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.588714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.589260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.589751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.591022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.595321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.643730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1020/2001] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:18.644887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.650623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbc53f197bc462d95d93e282b761e80', 'request_id': 'req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:18.663284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:18.664064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbc53f197bc462d95d93e282b761e80', 'request_id': 'req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:18.697943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbc53f197bc462d95d93e282b761e80', 'request_id': 'req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:18.718229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] The token cache is being invalidate because project 8dbc53f197bc462d95d93e282b761e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:18.718835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:18.720728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:18.722740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/2002] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:18 2022] DELETE /identity/v3/projects/8dbc53f197bc462d95d93e282b761e80 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:18.723845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.741653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:18.814386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f15da-6acf-4110-a743-3c57544c176f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.820132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7344c8-2d8d-40a6-a533-0939cbe44414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.822015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344c8-2d8d-40a6-a533-0939cbe44414 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.822547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344c8-2d8d-40a6-a533-0939cbe44414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.822979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344c8-2d8d-40a6-a533-0939cbe44414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.824065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344c8-2d8d-40a6-a533-0939cbe44414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.826146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344c8-2d8d-40a6-a533-0939cbe44414 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.827971 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:18.841898 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1021/2003] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:18.844288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.845805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.845805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.871985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfc9c7-5e24-4d75-a7e7-6908e0de5ef2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.879278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.880238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.880516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.880803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.881504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.883635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.885530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:18.888096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:18.910346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/2004] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:18.911060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.911580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1022/2005] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:18.918074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:18.920614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:18.936533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcd80f-b075-4336-bd96-a998f47b4a2c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.948218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25493f-eba7-4300-8d3c-cd3a1f03550e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:18.957646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.958522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38926c-815c-48ef-8e1f-52ac94b1236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:18.959254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38926c-815c-48ef-8e1f-52ac94b1236f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.959987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38926c-815c-48ef-8e1f-52ac94b1236f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.960240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38926c-815c-48ef-8e1f-52ac94b1236f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.961000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.961525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38926c-815c-48ef-8e1f-52ac94b1236f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.962169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.962832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.964846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38926c-815c-48ef-8e1f-52ac94b1236f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.965193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:18.971154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:18.990167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1023/2006] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:18.993227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/2007] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:18.993540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:18.993964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:18.994329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:18.994663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.007306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ca19f-8e71-4255-b6cc-0b45113d9eda admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.012681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.013413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.013725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.014003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] PATH_INFO: `/v3/projects/d9d7073d4e8947119b5e7cf0b985082a/users/b90e916b306b4a2e9157ed9778957da5/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.016016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:19.018150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:19.028695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1024/2008] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:19.030171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.036027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d7073d4e8947119b5e7cf0b985082a, user_id=b90e916b306b4a2e9157ed9778957da5, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.039154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.044492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66ebc-6648-4cf3-a451-504484b51047 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.051618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.053136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.053495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.053941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] PATH_INFO: `/v3/projects/78479f7dc6dc44c19e6090b122f83960/users/31a811935b4b4b58aaf3c626d71b70d3/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.058806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/2009] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:18 2022] PUT /identity/v3/projects/d9d7073d4e8947119b5e7cf0b985082a/users/b90e916b306b4a2e9157ed9778957da5/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:19.060375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.083406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78479f7dc6dc44c19e6090b122f83960, user_id=31a811935b4b4b58aaf3c626d71b70d3, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.086470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.104810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1025/2010] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:19 2022] PUT /identity/v3/projects/78479f7dc6dc44c19e6090b122f83960/users/31a811935b4b4b58aaf3c626d71b70d3/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:19.106065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.155435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d4ea-e1b7-4550-8451-14b9b8dc9634 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.192969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52437a38-6b27-4bd5-b232-1df76c9b4a13 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.198140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95442d1-5b7b-418a-9469-82e02a3c9479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.198911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95442d1-5b7b-418a-9469-82e02a3c9479 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.199247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95442d1-5b7b-418a-9469-82e02a3c9479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.199530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95442d1-5b7b-418a-9469-82e02a3c9479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.200226 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95442d1-5b7b-418a-9469-82e02a3c9479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.201920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95442d1-5b7b-418a-9469-82e02a3c9479 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.218830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1026/2011] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:19.221834 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e4374-72e3-4be0-9f21-a670e38d772e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.222237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e4374-72e3-4be0-9f21-a670e38d772e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.222849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e4374-72e3-4be0-9f21-a670e38d772e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.254084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.255073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.255509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.255962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.257065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:19.257385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.259067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.265124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-253e4374-72e3-4be0-9f21-a670e38d772e None None] MFA Rules not processed for user `31a811935b4b4b58aaf3c626d71b70d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:19.266465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-253e4374-72e3-4be0-9f21-a670e38d772e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:19.272512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/2012] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:19.273421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.285084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871368f-ef10-4758-9ab7-cdadda0dfc94 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.290372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.291075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.291398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.291690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] PATH_INFO: `/v3/projects/d9d7073d4e8947119b5e7cf0b985082a/users/b90e916b306b4a2e9157ed9778957da5/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.293827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d7073d4e8947119b5e7cf0b985082a, user_id=b90e916b306b4a2e9157ed9778957da5, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.296644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.311537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/2013] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:19 2022] PUT /identity/v3/projects/d9d7073d4e8947119b5e7cf0b985082a/users/b90e916b306b4a2e9157ed9778957da5/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:19.312175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.327015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:19.358335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:19.359368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1027/2014] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:19.362832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.363718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.364514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.429857 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e84b76-8669-43d3-b0fd-a34eddf6774a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.435493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] MFA Rules not processed for user `b90e916b306b4a2e9157ed9778957da5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:19.437018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.438073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.439183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.440141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.440894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.442915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.445215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:19.488400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/2015] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:19 2022] POST /identity/v3/projects => generated 359 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:19.490098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.493605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:19.575019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1028/2016] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:19.576713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.596504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b3ed4-410b-42c9-87a2-ae87f593bba9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.665918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687373-7bb3-4d9a-9a39-38613e57b6df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.671190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.672203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.672500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.673404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.716761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.717847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.718398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.719887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.721163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.725451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.758821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.760802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.773950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1029/2017] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:19 2022] POST /identity/v3/users => generated 280 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:19.776236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.777980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/2018] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:19.779010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.779741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:19.794916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75139caf-f2ab-46c6-9458-9a1c6a9dd90e service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.801905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.802624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.802901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.803268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.804330 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.806282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.823994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/2019] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:19.825073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.853204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ecb4f-76db-41a8-8180-ed79052a1e9d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.867796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.868411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.868707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.869124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.869849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.874718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.901826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/2020] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:19.905266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:19.919722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337691-afc7-4456-8a21-34174ffb713e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.927695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f0ec4-59aa-4a82-8049-8afd2382a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.928431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f0ec4-59aa-4a82-8049-8afd2382a22c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.928883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f0ec4-59aa-4a82-8049-8afd2382a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.929174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f0ec4-59aa-4a82-8049-8afd2382a22c admin admin] PATH_INFO: `/v3/projects/332bb219c4d84745b3d557a34e97077e/users/290394f17f4a44a28d53a97a4c501434/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.953680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6824-89fa-4b04-90e0-7f36fa474e48 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:19.956100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:19.965037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f0ec4-59aa-4a82-8049-8afd2382a22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=332bb219c4d84745b3d557a34e97077e, user_id=290394f17f4a44a28d53a97a4c501434, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:19.975021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f0ec4-59aa-4a82-8049-8afd2382a22c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:19.986452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:19.987127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:19.987446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:19.987728 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:19.989910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:20.008726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/2021] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:19 2022] PUT /identity/v3/projects/332bb219c4d84745b3d557a34e97077e/users/290394f17f4a44a28d53a97a4c501434/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:20.023766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:20.024781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:20.025845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:20.028006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:20.125056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] MFA Rules not processed for user `290394f17f4a44a28d53a97a4c501434`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:20.133924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:20.147652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:20.149654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:20.167741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1030/2022] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:20.389530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/2023] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:20 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:20.739004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:20.752227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba0a09-ccff-449e-b94c-98ca58760278 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:20.879134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b022f2-cddf-4cb5-bc29-fd7df689699d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:20.879836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b022f2-cddf-4cb5-bc29-fd7df689699d service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:20.880126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b022f2-cddf-4cb5-bc29-fd7df689699d service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:20.880473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b022f2-cddf-4cb5-bc29-fd7df689699d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:20.893946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b022f2-cddf-4cb5-bc29-fd7df689699d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:20.896273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b022f2-cddf-4cb5-bc29-fd7df689699d service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:20.929938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1031/2024] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:20 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:20.930683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:21.017048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94836a-d722-45f0-a97a-413d7c790bc9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:21.022723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:21.023546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:21.023897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:21.024244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] PATH_INFO: `/v3/users/290394f17f4a44a28d53a97a4c501434` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:21.025598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=290394f17f4a44a28d53a97a4c501434)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:21.027701 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:21.116694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290394f17f4a44a28d53a97a4c501434'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:21.117855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290394f17f4a44a28d53a97a4c501434'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:21.131316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290394f17f4a44a28d53a97a4c501434'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:21.136208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290394f17f4a44a28d53a97a4c501434'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:21.192650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290394f17f4a44a28d53a97a4c501434'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:21.224585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:21.245779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:21.246320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:21.246729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:21.250018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/2025] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:20 2022] DELETE /identity/v3/users/290394f17f4a44a28d53a97a4c501434 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:21.270723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:21.271232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:21.383121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1032/2026] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:21.406586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:21.410063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:21.544825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0701b-2dd0-489e-a017-2a9ca524eaa5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:21.554368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:21.554368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:21.554368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:21.554368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:21.568194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:21.570069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:21.583920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/2027] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:22.223157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:22.224949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:22.238334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feac4-9171-49f7-b590-e11ad8d89274 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:22.244102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:22.245075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:22.245564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:22.246424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] PATH_INFO: `/v3/projects/332bb219c4d84745b3d557a34e97077e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:22.259290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=332bb219c4d84745b3d557a34e97077e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:22.262070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:22.302664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:22.305064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:22.373739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332bb219c4d84745b3d557a34e97077e', 'request_id': 'req-e67c9a6e-207a-4115-9471-d17d01d5ee3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:22.380241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:22.399381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332bb219c4d84745b3d557a34e97077e', 'request_id': 'req-e67c9a6e-207a-4115-9471-d17d01d5ee3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:22.404451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332bb219c4d84745b3d557a34e97077e', 'request_id': 'req-e67c9a6e-207a-4115-9471-d17d01d5ee3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:22.420943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] The token cache is being invalidate because project 332bb219c4d84745b3d557a34e97077e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:22.421517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:22.422740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e67c9a6e-207a-4115-9471-d17d01d5ee3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:22.429375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1033/2028] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:22 2022] DELETE /identity/v3/projects/332bb219c4d84745b3d557a34e97077e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:22.432006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ec9e-8156-4ba1-83a0-62f96fee0954 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:22.432504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ec9e-8156-4ba1-83a0-62f96fee0954 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:22.433243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ec9e-8156-4ba1-83a0-62f96fee0954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:22.464794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e5fc-b5fa-41d3-8c61-6110ece9fa98 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:22.477166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4f39ec9e-8156-4ba1-83a0-62f96fee0954 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:22.480400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4f39ec9e-8156-4ba1-83a0-62f96fee0954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:22.575572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:22.576244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:22.576526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:22.576842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:22.580913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:22.684670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1034/2029] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:22.685628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:22.685628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:22.685952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:22.698965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:22.702475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:22.731754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/2030] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:22.738773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:22.744517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:22.772139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1035/2031] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:22.790576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:22.811545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dcd85-d001-431c-b2bf-7ca03b1afbc2 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:22.820439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:22.821473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:22.821963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:22.822441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:22.823716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:22.824262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:22.827522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:22.845920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:22.863812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/2032] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:22.864479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:22.885808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daa587-5649-4d42-b342-bf2ed495f1f9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:22.901857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:22.904594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:22.905228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:22.905734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:22.907648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:22.914354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:23.012393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1036/2033] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:22 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:23.021257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:23.045672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ffd7c-a079-40f7-9d31-c21ec3c41704 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:23.213794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:23.215085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:23.215686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:23.217518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:23.218914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:23.224670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:23.280481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/2034] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:23 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:23.281464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:23.305236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d9d66-1ede-45ed-8e5b-e1b2515673f1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:23.319932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:23.319932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:23.319932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:23.320568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:23.322222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:23.326964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:23.383860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1037/2035] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:23.385070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:23.438500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635803cf-dea0-46dc-85ee-0e6771b484b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:23.472003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95de12-048b-492d-9e89-82c7bf352b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:23.475214 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95de12-048b-492d-9e89-82c7bf352b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:23.476940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95de12-048b-492d-9e89-82c7bf352b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:23.481533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95de12-048b-492d-9e89-82c7bf352b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:23.488652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95de12-048b-492d-9e89-82c7bf352b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:23.499379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95de12-048b-492d-9e89-82c7bf352b4e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:23.533745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/2036] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:23.539396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:23.573776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b1003-80f6-445e-9757-4c339ed42fdf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:23.588359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:23.590687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:23.591025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:23.592161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] PATH_INFO: `/v3/projects/1eee1f31b62e486eb31b107e7df27749/users/9822ed443a8244329ea78b47028b9eda/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:23.645128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eee1f31b62e486eb31b107e7df27749, user_id=9822ed443a8244329ea78b47028b9eda, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:23.653797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:23.701784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1038/2037] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:23 2022] PUT /identity/v3/projects/1eee1f31b62e486eb31b107e7df27749/users/9822ed443a8244329ea78b47028b9eda/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:23.724996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:23.726108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:23.726744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:23.841413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] MFA Rules not processed for user `9822ed443a8244329ea78b47028b9eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:23.843876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:24.145389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/2038] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:24.186577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:24.191522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:24.243210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:24.251584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:24.330173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfd89d-132c-4682-a991-1b8149f86be9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:24.334899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:24.335577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:24.335862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:24.337439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:24.352587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:24.354703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:24.381914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1039/2039] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:24.400174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cbd52-03e7-4510-948c-0d46325b234d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:24.405734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:24.407565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:24.407565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:24.408054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:24.412657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:24.445010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:24.461528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:24.520677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:24.554370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:24.556081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b402c-ab7e-4722-a11c-4a2a3fb2d46a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:24.570383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/2040] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:24.640108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd9366-f1be-40dd-8fc2-4e32c7b3a67a service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:24.652538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:24.653941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:24.654768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:24.655373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] PATH_INFO: `/v3/users/9822ed443a8244329ea78b47028b9eda` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:24.659727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9822ed443a8244329ea78b47028b9eda)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:24.666563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:24.801759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9822ed443a8244329ea78b47028b9eda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:24.820340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9822ed443a8244329ea78b47028b9eda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:24.822504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9822ed443a8244329ea78b47028b9eda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:24.824427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9822ed443a8244329ea78b47028b9eda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:24.871801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9822ed443a8244329ea78b47028b9eda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:24.949860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1040/2041] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:24 2022] DELETE /identity/v3/users/9822ed443a8244329ea78b47028b9eda => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:24.952585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:24.953382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:24.954678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:24.998783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:25.001021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:25.132798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/2042] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:25.159687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:25.169476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:25.355581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d3c28-afef-424a-8399-859bb3aeb899 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:25.363766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:25.364844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:25.365571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:25.366037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:25.381927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:25.384876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:25.415503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1041/2043] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:26.125742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:26.127172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:26.149139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84c411-24c3-4752-a362-ebc36485955f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:26.170193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:26.171708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:26.172178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:26.173173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] PATH_INFO: `/v3/projects/1eee1f31b62e486eb31b107e7df27749` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:26.177884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eee1f31b62e486eb31b107e7df27749)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:26.180245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:26.318365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eee1f31b62e486eb31b107e7df27749', 'request_id': 'req-84a258b1-754f-4e07-8dbf-cb588eac8a61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:26.339451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eee1f31b62e486eb31b107e7df27749', 'request_id': 'req-84a258b1-754f-4e07-8dbf-cb588eac8a61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:26.395685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eee1f31b62e486eb31b107e7df27749', 'request_id': 'req-84a258b1-754f-4e07-8dbf-cb588eac8a61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:26.414969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] The token cache is being invalidate because project 1eee1f31b62e486eb31b107e7df27749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:26.415480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:26.418228 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-84a258b1-754f-4e07-8dbf-cb588eac8a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:26.443498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2044] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:26 2022] DELETE /identity/v3/projects/1eee1f31b62e486eb31b107e7df27749 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:29.512740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:29.514592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:29.532101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60568817-e2ad-4140-9c5e-800d899dfbb9 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:29.622342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:29.655774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:29.657004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:29.657308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:29.657688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] PATH_INFO: `/v3/users/08b315584bc84bc188108633d3f3d5f3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:29.659510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b315584bc84bc188108633d3f3d5f3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:29.661553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:29.751656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b315584bc84bc188108633d3f3d5f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:29.767046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b315584bc84bc188108633d3f3d5f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:29.767953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b315584bc84bc188108633d3f3d5f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:29.768881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b315584bc84bc188108633d3f3d5f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:29.790632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b315584bc84bc188108633d3f3d5f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:29.808969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1042/2045] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:29 2022] DELETE /identity/v3/users/08b315584bc84bc188108633d3f3d5f3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:29.815154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:29.816177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:29.816177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:29.846405 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:29.848826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:29.915671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:29.952506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2046] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:29.966142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:29.968087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:30.080906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70c63f-0f30-4269-8da7-eab1a005c28c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:30.088476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:30.091009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:30.091275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:30.091807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:30.112895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:30.116773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:30.139589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1043/2047] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:30.676915 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:30.679525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:30.702996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8a944-7f40-41ed-a01c-e1e21fb7731b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:30.713218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:30.714120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:30.714822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:30.715200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] PATH_INFO: `/v3/projects/c8e9dbbd088b43d68a5083138766251b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:30.717234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e9dbbd088b43d68a5083138766251b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:30.720459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:30.815891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e9dbbd088b43d68a5083138766251b', 'request_id': 'req-2795b949-bae7-4073-bc83-eda87f7e75cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:30.830564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e9dbbd088b43d68a5083138766251b', 'request_id': 'req-2795b949-bae7-4073-bc83-eda87f7e75cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:30.843621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e9dbbd088b43d68a5083138766251b', 'request_id': 'req-2795b949-bae7-4073-bc83-eda87f7e75cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:30.862619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] The token cache is being invalidate because project c8e9dbbd088b43d68a5083138766251b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:30.862879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:30.863918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:30.866463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2048] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:30 2022] DELETE /identity/v3/projects/c8e9dbbd088b43d68a5083138766251b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:30.871088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:30.885055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ef5ac-522c-4dac-b48f-de2d87f5562d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:31.024050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:31.024731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:31.025026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:31.025311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] PATH_INFO: `/v3/users/b080f56558b2404ebb92ed210715674f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:31.026687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b080f56558b2404ebb92ed210715674f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:31.028770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:31.038322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:31.111685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b080f56558b2404ebb92ed210715674f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:31.121734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b080f56558b2404ebb92ed210715674f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:31.122433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b080f56558b2404ebb92ed210715674f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:31.123086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b080f56558b2404ebb92ed210715674f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:31.145307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b080f56558b2404ebb92ed210715674f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:31.173781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1044/2049] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:30 2022] DELETE /identity/v3/users/b080f56558b2404ebb92ed210715674f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:31.697135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:31.699662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:31.728425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795b949-bae7-4073-bc83-eda87f7e75cc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:31.904384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:31.905584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:31.906934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:31.907481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] PATH_INFO: `/v3/projects/a1fd58000bd549d2a45d6d6c61f1225e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:31.909964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fd58000bd549d2a45d6d6c61f1225e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:31.914122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:32.132568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fd58000bd549d2a45d6d6c61f1225e', 'request_id': 'req-d2fabf41-faec-408b-8398-2fb616db0ae2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.141281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fd58000bd549d2a45d6d6c61f1225e', 'request_id': 'req-d2fabf41-faec-408b-8398-2fb616db0ae2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.156109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fd58000bd549d2a45d6d6c61f1225e', 'request_id': 'req-d2fabf41-faec-408b-8398-2fb616db0ae2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.182722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:32.207972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] The token cache is being invalidate because project a1fd58000bd549d2a45d6d6c61f1225e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:32.208285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.209969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fabf41-faec-408b-8398-2fb616db0ae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.211984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2050] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:31 2022] DELETE /identity/v3/projects/a1fd58000bd549d2a45d6d6c61f1225e => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:32.216711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:32.219801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:32.244599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e944c0-6123-408c-9f76-562d2a873865 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:32.336929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:32.337616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:32.337975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:32.338280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] PATH_INFO: `/v3/users/b90e916b306b4a2e9157ed9778957da5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:32.339596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90e916b306b4a2e9157ed9778957da5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:32.341627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:32.435653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90e916b306b4a2e9157ed9778957da5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.459037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90e916b306b4a2e9157ed9778957da5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.460746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90e916b306b4a2e9157ed9778957da5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.462236 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90e916b306b4a2e9157ed9778957da5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.488769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0377d24e-5e9c-4345-a64e-073b3f06f9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90e916b306b4a2e9157ed9778957da5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:32.506880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1045/2051] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:32 2022] DELETE /identity/v3/users/b90e916b306b4a2e9157ed9778957da5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:32.770652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb9408-3138-4355-84ef-c87ec3fa9158 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:32.771117 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb9408-3138-4355-84ef-c87ec3fa9158 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:32.771605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb9408-3138-4355-84ef-c87ec3fa9158 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:32.773600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2052] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:22:32.801927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:32.802208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:32.802551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:32.814485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:32.836799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] MFA Rules not processed for user `5df6c21c2ed44bf0b07cd16454b3e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:32.838552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:32.899589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb9408-3138-4355-84ef-c87ec3fa9158 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:32.901705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cefb9408-3138-4355-84ef-c87ec3fa9158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:32.981116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1046/2053] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:22:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:33.005871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.007364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.024923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999e534-e36a-44da-b463-2f0ac97051a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.026703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb9408-3138-4355-84ef-c87ec3fa9158 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.033386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.034115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.034278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.034537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.034904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] PATH_INFO: `/v3/users/720be309cbe14940873d774b06e91bf9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.035651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.036143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.036491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=720be309cbe14940873d774b06e91bf9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.036914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.039341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.057697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.060599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.095744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1047/2054] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:33.097289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.100962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.117613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720be309cbe14940873d774b06e91bf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.118594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720be309cbe14940873d774b06e91bf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.124327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f7aab-6e7d-4a6f-8196-b320bfb07817 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.130910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.131732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720be309cbe14940873d774b06e91bf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.132371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.132606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.132894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] PATH_INFO: `/v3/projects/d9d7073d4e8947119b5e7cf0b985082a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.133087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720be309cbe14940873d774b06e91bf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.134225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d7073d4e8947119b5e7cf0b985082a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.137068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.159624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5752b54b-3f60-4724-ba85-ea7da915cf7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720be309cbe14940873d774b06e91bf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.223811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2055] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:32 2022] DELETE /identity/v3/users/720be309cbe14940873d774b06e91bf9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:33.237878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d7073d4e8947119b5e7cf0b985082a', 'request_id': 'req-e26b63e1-7a32-415c-a9e3-8e74caef387f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.252189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d7073d4e8947119b5e7cf0b985082a', 'request_id': 'req-e26b63e1-7a32-415c-a9e3-8e74caef387f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.266262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d7073d4e8947119b5e7cf0b985082a', 'request_id': 'req-e26b63e1-7a32-415c-a9e3-8e74caef387f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.292117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] The token cache is being invalidate because project d9d7073d4e8947119b5e7cf0b985082a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:33.292520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.295692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e26b63e1-7a32-415c-a9e3-8e74caef387f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.299451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1048/2056] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:33 2022] DELETE /identity/v3/projects/d9d7073d4e8947119b5e7cf0b985082a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:33.306730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.307033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.307317 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.364122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:33.367304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.456076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2057] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:33.464756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.464756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.465018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.489775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:33.492694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.505390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.507179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1049/2058] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:33.510282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.511701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.526576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75bb48-acaf-4615-a8a3-943e91a49881 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.531688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d2716-f800-49af-bbf7-2ece4f2ac841 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.533771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.535205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.535742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.536102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] PATH_INFO: `/v3/projects/c87396f5e6bf46f5b85720c8e365da81` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.537808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87396f5e6bf46f5b85720c8e365da81)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.539994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.542041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.543135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.543690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.544396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.545949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:33.546507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.550155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.569980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:33.577646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1050/2059] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:22:33.583940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.596113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be481e31-1429-4607-8a45-ff266bc964da admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.601440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.602148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.602445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.602727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.603480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.605487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.640603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1051/2060] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:33.642014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87396f5e6bf46f5b85720c8e365da81', 'request_id': 'req-908070ca-d159-49b2-b19e-5a6c69fc8729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.645233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.654623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87396f5e6bf46f5b85720c8e365da81', 'request_id': 'req-908070ca-d159-49b2-b19e-5a6c69fc8729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.667113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87396f5e6bf46f5b85720c8e365da81', 'request_id': 'req-908070ca-d159-49b2-b19e-5a6c69fc8729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.688067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:33.691105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] The token cache is being invalidate because project c87396f5e6bf46f5b85720c8e365da81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:33.691670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.693052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-908070ca-d159-49b2-b19e-5a6c69fc8729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:33.699175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2061] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:33 2022] DELETE /identity/v3/projects/c87396f5e6bf46f5b85720c8e365da81 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:33.709746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6e8fd-9830-475b-a46d-9c09f81c90e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.710220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6e8fd-9830-475b-a46d-9c09f81c90e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.710718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6e8fd-9830-475b-a46d-9c09f81c90e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.742921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dcf6e8fd-9830-475b-a46d-9c09f81c90e4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:33.744306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dcf6e8fd-9830-475b-a46d-9c09f81c90e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.753072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2062] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:33.761418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce3dd1-b583-4820-91e2-bf78ba85986e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.761724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.761973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.762364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.770408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.771606 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.772125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.772640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.773912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.777754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.785271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:33.786594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.795265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2063] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:33.801628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.818629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331631-2281-4cb3-ad01-172c53fa7719 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.820770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1052/2064] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:33.826059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.829368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:33.831008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.831975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.832389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.832793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.833926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:33.834479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.837924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.846565 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf49f33-b175-414a-9c8c-cf40feff3047 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.853725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.854638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.854933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.856199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.856986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.860262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.868135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2065] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:22:33.869642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.879168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1053/2066] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:33.881215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.890611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a0a1b-908a-4ef7-bb85-e2bf854f4069 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.895885 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033b1a7-5fe8-4faa-af4d-20099aa4e94b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.901426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.901721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.902286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.902453 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.902625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.902914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.903068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.903677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.903843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.904957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.905403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.908891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.918295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1054/2067] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:33.923830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.935432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8d2c-9dc0-4bfa-94ca-b1d145a6ce75 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:33.942480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4de7f-6830-40ce-b2cb-173a5d7fe5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:33.943258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4de7f-6830-40ce-b2cb-173a5d7fe5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:33.943670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4de7f-6830-40ce-b2cb-173a5d7fe5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:33.943975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4de7f-6830-40ce-b2cb-173a5d7fe5f2 admin admin] PATH_INFO: `/v3/projects/4235f62a0be64bb5bbfde93883310b43/users/17cbfb4d9b0148e1a7090afe2f14735a/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:33.959439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2068] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:33.964589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:33.972125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4de7f-6830-40ce-b2cb-173a5d7fe5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4235f62a0be64bb5bbfde93883310b43, user_id=17cbfb4d9b0148e1a7090afe2f14735a, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:33.975555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4de7f-6830-40ce-b2cb-173a5d7fe5f2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:33.993837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1055/2069] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:33 2022] PUT /identity/v3/projects/4235f62a0be64bb5bbfde93883310b43/users/17cbfb4d9b0148e1a7090afe2f14735a/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:34.006695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.007073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.007538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.054398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] MFA Rules not processed for user `17cbfb4d9b0148e1a7090afe2f14735a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:34.056026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:34.071249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83e3ac-8127-4e74-9d88-6dd72d10f52d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.156825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:34.174028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1056/2070] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:34.174282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.174282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.174282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.174282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.175070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.180252 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.185816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.188903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:34.220809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2071] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:33 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:34.226595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.240762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37df09c-a8d8-49a4-8090-85feeb4c1e6c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.246992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.247996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.248458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.248917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.250665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.252919 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.266238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2072] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:34.270750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.273973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd6691-0143-4807-bbe6-d37fb5a8224a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.278124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.278780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.279821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.279821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.285715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9413c6-35e2-47b0-a5c8-9937d4809423 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.292696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a205f2-3c1a-43c3-b979-9c894f875e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.293580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a205f2-3c1a-43c3-b979-9c894f875e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.293757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.294035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a205f2-3c1a-43c3-b979-9c894f875e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.294408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a205f2-3c1a-43c3-b979-9c894f875e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.295226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a205f2-3c1a-43c3-b979-9c894f875e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.295523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.298280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a205f2-3c1a-43c3-b979-9c894f875e21 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.307867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1057/2073] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:34.313976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2074] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:34.318884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.326678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:34.340627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a38541-5d37-4268-b3f6-e43157d60520 service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.355110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.356381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.356861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.357343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] PATH_INFO: `/v3/projects/002ef70b05d34cbca6573a917c491150/users/aeae30992e834c1b9cafdb66d6b453df/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.394193 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=002ef70b05d34cbca6573a917c491150, user_id=aeae30992e834c1b9cafdb66d6b453df, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.400135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.427676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1058/2075] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:34 2022] PUT /identity/v3/projects/002ef70b05d34cbca6573a917c491150/users/aeae30992e834c1b9cafdb66d6b453df/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:34.447731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.448409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.449005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.494498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] MFA Rules not processed for user `aeae30992e834c1b9cafdb66d6b453df`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:34.495935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:34.602587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2076] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:34.608194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.737577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd2b2-d5be-4e1f-be75-05dae9d069aa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.746103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.747108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.747670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.748173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.749434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.753888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.805310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1059/2077] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:34 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:34.810029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.825019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581182ac-b2f7-45c1-85fb-8dbd07f16981 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.896643 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.897301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.897598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.897881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.898569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.900819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.929706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2078] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:34 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:34.931392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:34.944057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8d41-d716-4269-a54a-d0a7004127dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:34.951197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:34.951960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:34.952272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:34.952582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:34.965564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:34.965564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:34.982761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1060/2079] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:34.988230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:35.005553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc6a7c-37b7-4600-b1d9-6c62f6aeee3d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:35.013206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:35.014339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:35.014822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:35.015296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] PATH_INFO: `/v3/projects/319b0a77605d44699080dcb413848102/users/a83499189b3e4af09e00f4dc842b95dd/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:35.036580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319b0a77605d44699080dcb413848102, user_id=a83499189b3e4af09e00f4dc842b95dd, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:35.039791 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:35.057301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2080] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:34 2022] PUT /identity/v3/projects/319b0a77605d44699080dcb413848102/users/a83499189b3e4af09e00f4dc842b95dd/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:35.062420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:35.074484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6ddc2-7c58-4165-a4b3-94caa0aac608 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:35.148254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c9f7e-6d6b-4daf-9bdf-6b4abe04c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:35.148925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c9f7e-6d6b-4daf-9bdf-6b4abe04c40c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:35.149222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c9f7e-6d6b-4daf-9bdf-6b4abe04c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:35.149504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c9f7e-6d6b-4daf-9bdf-6b4abe04c40c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:35.150548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c9f7e-6d6b-4daf-9bdf-6b4abe04c40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:35.151941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c9f7e-6d6b-4daf-9bdf-6b4abe04c40c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:35.167431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1061/2081] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:35.173667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:35.182903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1cc5-2c9b-475e-8033-926c95517f98 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:35.191084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:35.191084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:35.191619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:35.192003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] PATH_INFO: `/v3/projects/319b0a77605d44699080dcb413848102/users/a83499189b3e4af09e00f4dc842b95dd/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:35.195394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319b0a77605d44699080dcb413848102, user_id=a83499189b3e4af09e00f4dc842b95dd, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:35.200849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:35.217813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2082] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:35 2022] PUT /identity/v3/projects/319b0a77605d44699080dcb413848102/users/a83499189b3e4af09e00f4dc842b95dd/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:35.230289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:35.230709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:35.230870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:35.264758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] MFA Rules not processed for user `a83499189b3e4af09e00f4dc842b95dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:35.266659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:35.317878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:35.353484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1062/2083] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:35.373473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:35.377748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:35.396512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:35.492293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4177e5-dc2a-4969-b463-23ede2c27858 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:35.499263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:35.500937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:35.501452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:35.501941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:35.515112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:35.517694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:35.530458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2084] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:35.642188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:35.660099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06285fa-d06f-4529-8f74-1ca2a4a57f57 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:35.667553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33df4c-486e-4abe-bd7a-ec54b00b8b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:35.668682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33df4c-486e-4abe-bd7a-ec54b00b8b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:35.669698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33df4c-486e-4abe-bd7a-ec54b00b8b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:35.670196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33df4c-486e-4abe-bd7a-ec54b00b8b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:35.787451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33df4c-486e-4abe-bd7a-ec54b00b8b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:35.789510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33df4c-486e-4abe-bd7a-ec54b00b8b13 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:35.808070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1063/2085] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:45.524894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=23 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:45.641047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:45.643091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:45.763589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85278a8-3765-48a7-a9d6-d3f6362acd8b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:45.775132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:45.777067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:45.777238 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:45.777765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] PATH_INFO: `/v3/users/31a811935b4b4b58aaf3c626d71b70d3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:45.780963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a811935b4b4b58aaf3c626d71b70d3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:45.784966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:45.801433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=39 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:45.872985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a811935b4b4b58aaf3c626d71b70d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:45.874149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a811935b4b4b58aaf3c626d71b70d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:45.892286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a811935b4b4b58aaf3c626d71b70d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:45.893360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a811935b4b4b58aaf3c626d71b70d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:45.920010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a811935b4b4b58aaf3c626d71b70d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:45.946626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2086] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:45 2022] DELETE /identity/v3/users/31a811935b4b4b58aaf3c626d71b70d3 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:45.965918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:45.967390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:45.968776 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:46.008483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:46.009925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:46.089079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1064/2087] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:46.104351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:46.106171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:46.183103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059c09-af36-44f4-a5ff-f0bcbf834054 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:46.187801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbf33d-0fab-4d6e-bb81-e7cf66c033e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:46.188379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbf33d-0fab-4d6e-bb81-e7cf66c033e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:46.188718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbf33d-0fab-4d6e-bb81-e7cf66c033e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:46.188945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbf33d-0fab-4d6e-bb81-e7cf66c033e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:46.199986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbf33d-0fab-4d6e-bb81-e7cf66c033e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:46.201501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbf33d-0fab-4d6e-bb81-e7cf66c033e0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:46.212557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2088] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:22:46.574474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:46.576532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:46.598971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab2588-531d-4a11-8a22-3dace0c07614 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:46.605544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:46.606585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:46.607016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:46.607479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] PATH_INFO: `/v3/projects/78479f7dc6dc44c19e6090b122f83960` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:46.610279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] RBAC: Authorizing `identity:delete_project(project_id=78479f7dc6dc44c19e6090b122f83960)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:46.614620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:46.692142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78479f7dc6dc44c19e6090b122f83960', 'request_id': 'req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:46.706740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78479f7dc6dc44c19e6090b122f83960', 'request_id': 'req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:46.717369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78479f7dc6dc44c19e6090b122f83960', 'request_id': 'req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:46.742286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] The token cache is being invalidate because project 78479f7dc6dc44c19e6090b122f83960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:46.742736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:46.743812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c77a53a2-9dbc-4b9a-be6f-9d72608a137c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:46.745368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1065/2089] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:46 2022] DELETE /identity/v3/projects/78479f7dc6dc44c19e6090b122f83960 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:46.752416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:46.753111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:46.753388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:46.788582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:46.790599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:46.822419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:46.918431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2090] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:46.929297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:46.930152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:46.930536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:46.960931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:46.962384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:46.977542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1066/2091] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:22:46.983232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.007033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9abf7-c3b8-4a58-9664-72e3811db919 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.013015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.014141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.014682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.015773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.017087 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:47.017563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.019639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.044639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2092] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:47.053192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.066539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74d634-cdf8-4392-83ad-34f0049192b6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.078404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.079232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.079796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.080286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.081330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.084604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.140473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1067/2093] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:47 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:47.146244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.164419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8beec9-f242-43c5-9885-31c3777d6860 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.271114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.271936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.272280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.272653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.273410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.275687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.304173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2094] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:47 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:47.308920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.321471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c821da-160d-44c2-8206-1ed62c2727be admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.327020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.327842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.328138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.328422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.329153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.330824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.342425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1068/2095] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:47.347073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.359665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971e43-a495-4867-a984-56c059500233 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.365541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf29648-6f3e-425b-b746-d148737f00bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.366253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29648-6f3e-425b-b746-d148737f00bf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.366596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29648-6f3e-425b-b746-d148737f00bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.366883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29648-6f3e-425b-b746-d148737f00bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.367705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf29648-6f3e-425b-b746-d148737f00bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.369593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf29648-6f3e-425b-b746-d148737f00bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.385959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2096] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:47.390797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.402401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48d822-0c76-42d5-ad23-c99013fa15cf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.409738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.410574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.410865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.411146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] PATH_INFO: `/v3/projects/3c05e2a1b9544d2f8fc6eafa6396b7df/users/0954ef6a9638487586cfe4ebf0bcb322/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.431268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c05e2a1b9544d2f8fc6eafa6396b7df, user_id=0954ef6a9638487586cfe4ebf0bcb322, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.434400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.450525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1069/2097] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:47 2022] PUT /identity/v3/projects/3c05e2a1b9544d2f8fc6eafa6396b7df/users/0954ef6a9638487586cfe4ebf0bcb322/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:47.464749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.465199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.465493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.501685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] MFA Rules not processed for user `0954ef6a9638487586cfe4ebf0bcb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:47.503463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:47.597239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2098] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:47.608730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:47.610841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:47.642116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:47.694821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada05f-2f3f-4754-979c-1e2cd121eee3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:47.701134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c061cb-0de6-404e-9ce2-04fa3b793b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:47.701958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c061cb-0de6-404e-9ce2-04fa3b793b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:47.702392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c061cb-0de6-404e-9ce2-04fa3b793b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:47.702846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c061cb-0de6-404e-9ce2-04fa3b793b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:47.713153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c061cb-0de6-404e-9ce2-04fa3b793b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:47.714663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c061cb-0de6-404e-9ce2-04fa3b793b78 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:47.726978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1070/2099] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:56.777491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:56.882376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:56.978158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf820c8-3030-44ac-ad0b-225165522aaf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:56.985127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:56.986023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:56.986364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:56.986650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] PATH_INFO: `/v3/users/0954ef6a9638487586cfe4ebf0bcb322` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:56.991823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0954ef6a9638487586cfe4ebf0bcb322)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:56.991823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:57.074542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0954ef6a9638487586cfe4ebf0bcb322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:57.075680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0954ef6a9638487586cfe4ebf0bcb322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:57.091660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0954ef6a9638487586cfe4ebf0bcb322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:57.093616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0954ef6a9638487586cfe4ebf0bcb322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:57.116563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0954ef6a9638487586cfe4ebf0bcb322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:57.132943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2100] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:22:56 2022] DELETE /identity/v3/users/0954ef6a9638487586cfe4ebf0bcb322 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:57.138102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:57.138648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:57.138648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:57.174884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:57.177187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:57.297177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1071/2101] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:57.315673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:57.317761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:57.394380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cb772-938a-4e30-8ac9-80dfcfa9e896 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:57.398478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fcb93-dab1-42e7-9193-ce400ba4f15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:57.399379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fcb93-dab1-42e7-9193-ce400ba4f15f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:57.399690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fcb93-dab1-42e7-9193-ce400ba4f15f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:57.400047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fcb93-dab1-42e7-9193-ce400ba4f15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:57.411286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fcb93-dab1-42e7-9193-ce400ba4f15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:57.412824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fcb93-dab1-42e7-9193-ce400ba4f15f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:57.424689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2102] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:22:57.966543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:57.970987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:58.022838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dd635-1677-4772-8812-e2a927c09347 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.038350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.040000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.040912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.041446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] PATH_INFO: `/v3/projects/3c05e2a1b9544d2f8fc6eafa6396b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.043949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c05e2a1b9544d2f8fc6eafa6396b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.048872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.152218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c05e2a1b9544d2f8fc6eafa6396b7df', 'request_id': 'req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:58.171899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c05e2a1b9544d2f8fc6eafa6396b7df', 'request_id': 'req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:58.189886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c05e2a1b9544d2f8fc6eafa6396b7df', 'request_id': 'req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:58.211053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] The token cache is being invalidate because project 3c05e2a1b9544d2f8fc6eafa6396b7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:22:58.211326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:58.212190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cbf99667-90c0-4596-8144-6fa4b6ba6fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:22:58.214294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1072/2103] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:22:57 2022] DELETE /identity/v3/projects/3c05e2a1b9544d2f8fc6eafa6396b7df => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:58.221241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.222272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.223034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.264615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:58.266982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:58.322961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:58.367535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2104] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:58.369956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.370498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.370946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.418626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:58.420851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:58.436529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1073/2105] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:58.441740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:58.461761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0771e-6348-4f9e-89a3-a83d38e905ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.484008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.485818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.486396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.487092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.488719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:22:58.489369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.494052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.526561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2106] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:22:58.527159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:58.538730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea8580-1f8a-4d04-9922-593fd6531424 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.547057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.547057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.547930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.548261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.555992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.555992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.596843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1074/2107] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:58 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:58.598814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:58.617636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6eeca9-fa05-4601-b249-1fa81ded1ed9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.707215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.707805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.708095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.708390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.709082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.711106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.738656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2108] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:58 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:58.744876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:58.762986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8aee-5439-4fdf-8681-1b867991b20a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.771848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.772926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.773423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.773875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.775080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.778066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.793028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1075/2109] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:58.798490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:58.809874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb8c0-9197-40a6-adff-6c0fbc2464ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.815741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f415ed-bb86-4d90-8d3e-07e11428bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.816425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f415ed-bb86-4d90-8d3e-07e11428bef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.816711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f415ed-bb86-4d90-8d3e-07e11428bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.817000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f415ed-bb86-4d90-8d3e-07e11428bef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.817718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f415ed-bb86-4d90-8d3e-07e11428bef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.819417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f415ed-bb86-4d90-8d3e-07e11428bef5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.833401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2110] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:58.837807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:58.850079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cabe4-1140-4531-a9ba-ff1a45677040 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:58.856883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:58.859749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.860254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.860906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] PATH_INFO: `/v3/projects/e2618b7967704714bdc546e905808388/users/770398957b484f50aeac69f4d8b8d1da/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.866536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:58.884413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2618b7967704714bdc546e905808388, user_id=770398957b484f50aeac69f4d8b8d1da, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:58.887317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:58.903378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1076/2111] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:58 2022] PUT /identity/v3/projects/e2618b7967704714bdc546e905808388/users/770398957b484f50aeac69f4d8b8d1da/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:22:58.915150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:58.915476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:58.915829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:58.950585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] MFA Rules not processed for user `770398957b484f50aeac69f4d8b8d1da`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:58.952975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:59.033193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2112] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:59.038272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.118686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6f4eb-8dd0-45e1-b7fb-2d14ec09b8f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.124749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.125544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.125924 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.126416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.127202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.129569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.164435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1077/2113] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:59 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:59.172064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.187560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ed38-764b-4b8a-afb7-a14de3b02be4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.254422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:22:59.313351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.314472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.315005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.315559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.316935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.321174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.370484 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.371998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2114] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:59 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:22:59.388148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9738-6bfd-42d7-b0ba-2fed931a10f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.396314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.397266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.397847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.398420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.399554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.402906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.427092 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1078/2115] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:22:59.436205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.456725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103bec3-22fd-430a-a6f4-80febb7f6184 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.464959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712aa87-52c8-42e3-ba9b-3a89dd89991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.466134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aa87-52c8-42e3-ba9b-3a89dd89991d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.466661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aa87-52c8-42e3-ba9b-3a89dd89991d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.467851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aa87-52c8-42e3-ba9b-3a89dd89991d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.470209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712aa87-52c8-42e3-ba9b-3a89dd89991d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.472095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712aa87-52c8-42e3-ba9b-3a89dd89991d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.495341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2116] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:22:59.496840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.508751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e30311-4095-467a-b1f1-be508eafdb63 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.513700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.514783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.515107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.515436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] PATH_INFO: `/v3/projects/efaf5bc199874bb2960e461bcaf60c49/users/205e3396f254405793c19ed00d802796/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.533918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efaf5bc199874bb2960e461bcaf60c49, user_id=205e3396f254405793c19ed00d802796, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.537051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.553561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1079/2117] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:22:59 2022] PUT /identity/v3/projects/efaf5bc199874bb2960e461bcaf60c49/users/205e3396f254405793c19ed00d802796/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:22:59.567100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.567499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.567905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.603989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] MFA Rules not processed for user `205e3396f254405793c19ed00d802796`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:22:59.605486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:22:59.687867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2118] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:22:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:22:59.693253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.773873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18417bcc-2a97-4bac-b320-53e2dab43008 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.782922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.784062 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.784609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.785161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.786836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.791033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.840711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1080/2119] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:22:59 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:59.842027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.855260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed396bb6-7eb2-43a8-9321-2b560ab337af None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.926707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.927467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.927772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.928175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.928943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.931242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.958118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2120] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:22:59 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:22:59.963114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:22:59.975668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede12a9c-9713-403c-bccb-1d05d2b80122 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:22:59.980910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:22:59.981559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:22:59.981846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:22:59.982122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:22:59.982964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:22:59.985210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:22:59.997512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1081/2121] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:23:00.002416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:00.013385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893252-2fb5-462b-beb3-51d46b627342 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:00.019750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:00.020479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:00.020769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:00.021049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] PATH_INFO: `/v3/projects/568fe34752e646f08fb0513473abdf0b/users/3710c20f2b164a6cabc14b3f0f381f90/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:00.039967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568fe34752e646f08fb0513473abdf0b, user_id=3710c20f2b164a6cabc14b3f0f381f90, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:00.042983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:00.060300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2122] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:00 2022] PUT /identity/v3/projects/568fe34752e646f08fb0513473abdf0b/users/3710c20f2b164a6cabc14b3f0f381f90/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:00.065418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:00.077519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dc15c-398d-4742-a40d-52ad78b4248f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:00.081902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:00.149240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54edda-df7a-4364-81a0-41e2428cc712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:00.149979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54edda-df7a-4364-81a0-41e2428cc712 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:00.150302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54edda-df7a-4364-81a0-41e2428cc712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:00.150653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54edda-df7a-4364-81a0-41e2428cc712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:00.151521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54edda-df7a-4364-81a0-41e2428cc712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:00.153380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54edda-df7a-4364-81a0-41e2428cc712 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:00.167249 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1082/2123] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:00.175216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:00.194741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a477c-18f5-40f4-aac4-f206f2d39537 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:00.202052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:00.202928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:00.203286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:00.203677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] PATH_INFO: `/v3/projects/568fe34752e646f08fb0513473abdf0b/users/3710c20f2b164a6cabc14b3f0f381f90/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:00.206152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568fe34752e646f08fb0513473abdf0b, user_id=3710c20f2b164a6cabc14b3f0f381f90, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:00.209510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:00.213737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:00.226193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2124] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:00 2022] PUT /identity/v3/projects/568fe34752e646f08fb0513473abdf0b/users/3710c20f2b164a6cabc14b3f0f381f90/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:00.239956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:00.240567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:00.241316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:00.281873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] MFA Rules not processed for user `3710c20f2b164a6cabc14b3f0f381f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:00.283213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:00.369034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1083/2125] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:00.383167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:00.385213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:00.455864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626ce81-896f-4979-8129-d4ba7f6938a0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:00.459733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:00.460401 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:00.460685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:00.460967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:00.564697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:00.566782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:00.581589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2126] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:23:06.665014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:06.675579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f17a7-49e6-4e27-a817-c05b8652ca01 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:06.679317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:06.680015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:06.680308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:06.680580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:06.776672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:06.777909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:06.791429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1084/2127] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:07.995520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:07.998322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:08.011666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cee17-196f-4fc5-a559-516d8d9a306e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:08.016822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6ffb-13ff-40c8-8125-ec275b4d1585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:08.017630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6ffb-13ff-40c8-8125-ec275b4d1585 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:08.018033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6ffb-13ff-40c8-8125-ec275b4d1585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:08.018319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6ffb-13ff-40c8-8125-ec275b4d1585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:08.030799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c6ffb-13ff-40c8-8125-ec275b4d1585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:08.032293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c6ffb-13ff-40c8-8125-ec275b4d1585 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:08.050248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2128] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:23:16.615063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:16.665048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:16.713465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d1a8a-5bb3-4968-b31a-67f57e22a0d9 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:16.721058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:16.721722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:16.722007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:16.722284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] PATH_INFO: `/v3/users/770398957b484f50aeac69f4d8b8d1da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:16.723601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770398957b484f50aeac69f4d8b8d1da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:16.725586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:16.804108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770398957b484f50aeac69f4d8b8d1da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:16.813767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770398957b484f50aeac69f4d8b8d1da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:16.815511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770398957b484f50aeac69f4d8b8d1da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:16.816156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770398957b484f50aeac69f4d8b8d1da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:16.833048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770398957b484f50aeac69f4d8b8d1da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:16.856291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1085/2129] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:16 2022] DELETE /identity/v3/users/770398957b484f50aeac69f4d8b8d1da => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:16.872456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:16.872960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:16.873299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:16.918278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:16.921156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:17.009635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2130] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:17.021272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:17.023403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:17.097999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0e419-540b-4907-b22f-94b23374b378 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:17.111059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:17.111916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:17.112276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:17.113121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:17.126037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:17.127599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:17.147533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1086/2131] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:17.742951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:17.744868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:17.760855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648a17b-2d08-4ee4-a204-3da19a986fab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:17.770203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:17.771010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:17.771544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:17.772031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] PATH_INFO: `/v3/projects/e2618b7967704714bdc546e905808388` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:17.773653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2618b7967704714bdc546e905808388)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:17.776647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:17.827809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:17.844072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2618b7967704714bdc546e905808388', 'request_id': 'req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:17.850730 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2618b7967704714bdc546e905808388', 'request_id': 'req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:17.857422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2618b7967704714bdc546e905808388', 'request_id': 'req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:17.880384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] The token cache is being invalidate because project e2618b7967704714bdc546e905808388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:17.881346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:17.881565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:17.883134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2132] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:17 2022] DELETE /identity/v3/projects/e2618b7967704714bdc546e905808388 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:17.887377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:17.898150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e72d1-5d2e-4087-84a4-835c62a497e1 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:17.956235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:17.974029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:17.974707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:17.974994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:17.975276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] PATH_INFO: `/v3/users/205e3396f254405793c19ed00d802796` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:17.976619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=205e3396f254405793c19ed00d802796)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:17.978646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:18.037882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205e3396f254405793c19ed00d802796'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.047709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205e3396f254405793c19ed00d802796'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.048394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205e3396f254405793c19ed00d802796'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.049025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205e3396f254405793c19ed00d802796'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.064893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205e3396f254405793c19ed00d802796'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.082350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1087/2133] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:17 2022] DELETE /identity/v3/users/205e3396f254405793c19ed00d802796 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:18.532554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:18.534685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:18.557742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df155d1-2fe5-4dde-b605-f8c01e8a5d1d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:18.647876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:18.648536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:18.648821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:18.649098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] PATH_INFO: `/v3/projects/efaf5bc199874bb2960e461bcaf60c49` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:18.650292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efaf5bc199874bb2960e461bcaf60c49)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:18.652270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:18.708973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efaf5bc199874bb2960e461bcaf60c49', 'request_id': 'req-39802a0b-2465-4db3-ba8d-81c2703c008f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.718236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efaf5bc199874bb2960e461bcaf60c49', 'request_id': 'req-39802a0b-2465-4db3-ba8d-81c2703c008f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.725509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efaf5bc199874bb2960e461bcaf60c49', 'request_id': 'req-39802a0b-2465-4db3-ba8d-81c2703c008f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.743185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] The token cache is being invalidate because project efaf5bc199874bb2960e461bcaf60c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:18.743518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.744384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.745991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2134] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:18 2022] DELETE /identity/v3/projects/efaf5bc199874bb2960e461bcaf60c49 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:18.751428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:18.753748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:18.773449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff93db-166a-45e1-91ca-04c2dc82bc4a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:18.855863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:18.857164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:18.857795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:18.858456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] PATH_INFO: `/v3/users/3710c20f2b164a6cabc14b3f0f381f90` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:18.860049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3710c20f2b164a6cabc14b3f0f381f90)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:18.862841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:18.963692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3710c20f2b164a6cabc14b3f0f381f90'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.976295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3710c20f2b164a6cabc14b3f0f381f90'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.977038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3710c20f2b164a6cabc14b3f0f381f90'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:18.977722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3710c20f2b164a6cabc14b3f0f381f90'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.007225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:19.010010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-84b24e9c-2db9-4ba6-8486-1068e9706750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3710c20f2b164a6cabc14b3f0f381f90'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.038621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1088/2135] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:18 2022] DELETE /identity/v3/users/3710c20f2b164a6cabc14b3f0f381f90 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:19.480985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:19.483170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:19.532288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39802a0b-2465-4db3-ba8d-81c2703c008f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:19.621217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:19.645280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:19.646706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:19.647150 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:19.647490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] PATH_INFO: `/v3/projects/568fe34752e646f08fb0513473abdf0b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:19.649661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568fe34752e646f08fb0513473abdf0b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:19.652825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:19.753008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568fe34752e646f08fb0513473abdf0b', 'request_id': 'req-e0e1786b-893d-41d3-8cd4-55f9cf166f18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.761541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568fe34752e646f08fb0513473abdf0b', 'request_id': 'req-e0e1786b-893d-41d3-8cd4-55f9cf166f18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.769756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568fe34752e646f08fb0513473abdf0b', 'request_id': 'req-e0e1786b-893d-41d3-8cd4-55f9cf166f18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.789612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] The token cache is being invalidate because project 568fe34752e646f08fb0513473abdf0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:19.790154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.793006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0e1786b-893d-41d3-8cd4-55f9cf166f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:19.800327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2136] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:19 2022] DELETE /identity/v3/projects/568fe34752e646f08fb0513473abdf0b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:19.804141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:19.804644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:19.805362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:19.853434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:19.861155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:19.983685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1089/2137] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:19.986247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:19.986544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:19.986830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.010053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:20.011689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:20.022329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2138] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:20.028645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.061243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0326d-e393-4066-8ff0-f40830a58c72 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.073179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.073842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.074209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.074536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.076630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:23:20.076630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.077906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.111167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1090/2139] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:23:20.115471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.127397 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a780b-5cc9-4303-a460-8a8de5ce6a55 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.133268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.134161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.134522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.134876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.135800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.138660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.184195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2140] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:20 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:20.188679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.201879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fd69a-b2a4-4889-a19d-1a856217d00d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.297156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.297850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.298176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.298534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.299556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.301890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.329135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1091/2141] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:20 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:23:20.333111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.345111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d58102-dcab-4a26-94fc-4f975a029fbd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.351020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.351720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.352008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.352388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.353149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.354836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.382512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2142] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:23:20.386831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.399242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e0039-2b7b-4730-b919-d6ce6a4a58c6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.406115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba302-8f63-40e5-a794-2a112a0033eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.407221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba302-8f63-40e5-a794-2a112a0033eb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.408563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba302-8f63-40e5-a794-2a112a0033eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.409054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba302-8f63-40e5-a794-2a112a0033eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.409954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba302-8f63-40e5-a794-2a112a0033eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.412509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba302-8f63-40e5-a794-2a112a0033eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.438880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1092/2143] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:20.440493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.454924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807fc01-7045-44d3-9081-7364434ed167 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.463679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.463679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.463679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.463679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] PATH_INFO: `/v3/projects/4804e980ecc84e12a90f01bca1acee0a/users/073b335a6ba0400e83608e8a862904ee/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.485247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4804e980ecc84e12a90f01bca1acee0a, user_id=073b335a6ba0400e83608e8a862904ee, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.488262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.509852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2144] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:20 2022] PUT /identity/v3/projects/4804e980ecc84e12a90f01bca1acee0a/users/073b335a6ba0400e83608e8a862904ee/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:20.608572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6a45b-9e1c-46ef-93f2-729b801fac47 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.609093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6a45b-9e1c-46ef-93f2-729b801fac47 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.609646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6a45b-9e1c-46ef-93f2-729b801fac47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.703705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cfe6a45b-9e1c-46ef-93f2-729b801fac47 None None] MFA Rules not processed for user `073b335a6ba0400e83608e8a862904ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:20.705735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cfe6a45b-9e1c-46ef-93f2-729b801fac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:20.820731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:20.851328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1093/2145] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:20.864652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:20.866688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:20.945061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a880c0-4c1a-496d-a26a-21ed15372d51 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:20.949353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ff62-ae84-402b-89fa-629121f0ed5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:20.950703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ff62-ae84-402b-89fa-629121f0ed5f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:20.951048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ff62-ae84-402b-89fa-629121f0ed5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:20.951649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ff62-ae84-402b-89fa-629121f0ed5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:20.965007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ff62-ae84-402b-89fa-629121f0ed5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:20.967958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ff62-ae84-402b-89fa-629121f0ed5f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:20.979926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2146] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:21.693107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:21.694041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:21.694041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:21.717319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] MFA Rules not processed for user `3c60b3a983f14f50b1b762a3954a1059`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:21.718557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:21.781494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1094/2147] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:24.662829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6571f-9c54-4b11-8800-3ad01ebeaf34 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:24.663463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6571f-9c54-4b11-8800-3ad01ebeaf34 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:24.663463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6571f-9c54-4b11-8800-3ad01ebeaf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:24.687159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72d6571f-9c54-4b11-8800-3ad01ebeaf34 None None] MFA Rules not processed for user `073b335a6ba0400e83608e8a862904ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:24.689415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72d6571f-9c54-4b11-8800-3ad01ebeaf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:24.697110 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2148] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:25.097759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:25.099656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:25.270949 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e58f6-29fc-4e6c-99a1-c5e0ea6d1598 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:25.288868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:25.289854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:25.290241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:25.290694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] PATH_INFO: `/v3/users/17cbfb4d9b0148e1a7090afe2f14735a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:25.292214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17cbfb4d9b0148e1a7090afe2f14735a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:25.295474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:25.401648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17cbfb4d9b0148e1a7090afe2f14735a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:25.417646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17cbfb4d9b0148e1a7090afe2f14735a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:25.419971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17cbfb4d9b0148e1a7090afe2f14735a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:25.420978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17cbfb4d9b0148e1a7090afe2f14735a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:25.445243 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17cbfb4d9b0148e1a7090afe2f14735a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:25.491628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1095/2149] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:25 2022] DELETE /identity/v3/users/17cbfb4d9b0148e1a7090afe2f14735a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:25.494828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:25.495795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:25.497449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:25.578454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:25.591391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:25.619056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:25.775618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2150] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:25.792825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:25.797671 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:25.817399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:25.927790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f3660-f86b-4905-bcf2-3f21dfb980e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:25.931986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbdd89-cfa0-4a45-bc3c-bdf910c7eb2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:25.933417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbdd89-cfa0-4a45-bc3c-bdf910c7eb2f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:25.934227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbdd89-cfa0-4a45-bc3c-bdf910c7eb2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:25.934910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbdd89-cfa0-4a45-bc3c-bdf910c7eb2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:25.954587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbdd89-cfa0-4a45-bc3c-bdf910c7eb2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:25.956317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbdd89-cfa0-4a45-bc3c-bdf910c7eb2f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:25.980354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1096/2151] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:26.515601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:26.517125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:26.538647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ab5aa-7896-430c-a2ee-11b8595a04b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:26.549985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:26.551322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:26.551913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:26.552459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] PATH_INFO: `/v3/projects/4235f62a0be64bb5bbfde93883310b43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:26.556657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4235f62a0be64bb5bbfde93883310b43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:26.561138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:26.662663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4235f62a0be64bb5bbfde93883310b43', 'request_id': 'req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:26.680189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4235f62a0be64bb5bbfde93883310b43', 'request_id': 'req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:26.693283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4235f62a0be64bb5bbfde93883310b43', 'request_id': 'req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:26.734459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] The token cache is being invalidate because project 4235f62a0be64bb5bbfde93883310b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:26.736159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:26.740060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dccef85-cac9-4ad9-b1a3-bb4db378efb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:26.745841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2152] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:26 2022] DELETE /identity/v3/projects/4235f62a0be64bb5bbfde93883310b43 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:26.754527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:26.755713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:26.757291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:26.828455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:26.832617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:26.964858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1097/2153] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:26.976766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:26.977379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:26.978140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.015521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:27.016817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:27.032666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2154] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:27.040043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.053274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deda991-a13c-4d91-9cfb-c34cd3e112bc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:27.058528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:27.059769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.060099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.060322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.061074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:23:27.061389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:27.063203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:27.080860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1098/2155] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:23:27.085326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.098122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4965-8e46-448b-8163-b50d892c7d30 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:27.104065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:27.104725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.105041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.105389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.106619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:27.108894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:27.142996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2156] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:27 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:27.147934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.160360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f7e0c-de54-408c-8e93-cc2c374729b5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:27.245358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:27.246119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.246517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.246913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.247742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:27.249936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:27.290012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1099/2157] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:27 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:27.293077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.333880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad54b5-9898-43dd-88a1-fb5b4bdc80c1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:27.355936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:27.363270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.363793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.364286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.365649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:27.373850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:27.437645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.438956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2158] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:27.452786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c51623-4cdb-44e6-8fd7-e2a38ca781a7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:27.459816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300cf7c-a2e8-4ece-b37c-a81eaf9acb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:27.460310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cf7c-a2e8-4ece-b37c-a81eaf9acb83 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.460727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cf7c-a2e8-4ece-b37c-a81eaf9acb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.461119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cf7c-a2e8-4ece-b37c-a81eaf9acb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.461969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300cf7c-a2e8-4ece-b37c-a81eaf9acb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:27.463870 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300cf7c-a2e8-4ece-b37c-a81eaf9acb83 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:27.481465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1100/2159] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:27.487305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.512825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c8576-ec1b-494f-84af-2274dc1b1fff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:27.522388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:27.524237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.524982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.525710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] PATH_INFO: `/v3/projects/1f75331a524c4a5a981dfd1fdc361467/users/1690be2d7032472391fa96e23e0c1bf0/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.589891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f75331a524c4a5a981dfd1fdc361467, user_id=1690be2d7032472391fa96e23e0c1bf0, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:27.589891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:27.612528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2160] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:27 2022] PUT /identity/v3/projects/1f75331a524c4a5a981dfd1fdc361467/users/1690be2d7032472391fa96e23e0c1bf0/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:27.617786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:27.618087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:27.618375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:27.653634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:27.656399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:27.678364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:27.849480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1101/2161] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:27.898527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:27.903604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:28.033001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:28.081731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60240725-35c9-4dc2-a26c-6f5a7348892f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:28.085882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495c267-1660-4ea1-a13c-627b2fa22ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:28.086527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495c267-1660-4ea1-a13c-627b2fa22ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:28.086815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495c267-1660-4ea1-a13c-627b2fa22ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:28.087306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495c267-1660-4ea1-a13c-627b2fa22ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:28.099923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495c267-1660-4ea1-a13c-627b2fa22ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:28.101625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495c267-1660-4ea1-a13c-627b2fa22ec9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:28.113774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2162] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:28.146662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:28.148819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:28.163985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46c50b-a92e-41bb-88d4-ba1eb7e35ff8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:28.172405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:28.172914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:28.173846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:28.173846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] PATH_INFO: `/v3/users/aeae30992e834c1b9cafdb66d6b453df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:28.175128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeae30992e834c1b9cafdb66d6b453df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:28.178411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:28.252972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeae30992e834c1b9cafdb66d6b453df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:28.265635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeae30992e834c1b9cafdb66d6b453df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:28.267031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeae30992e834c1b9cafdb66d6b453df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:28.268595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeae30992e834c1b9cafdb66d6b453df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:28.296065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeae30992e834c1b9cafdb66d6b453df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:28.331262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1102/2163] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:28 2022] DELETE /identity/v3/users/aeae30992e834c1b9cafdb66d6b453df => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:28.334368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f935fdd-ada4-4c28-a252-bbb0502c9572 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:28.335634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f935fdd-ada4-4c28-a252-bbb0502c9572 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:28.336329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f935fdd-ada4-4c28-a252-bbb0502c9572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:28.376328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f935fdd-ada4-4c28-a252-bbb0502c9572 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:28.378743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f935fdd-ada4-4c28-a252-bbb0502c9572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:28.536246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2164] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:28.570672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:28.574347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:28.711738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff684c-e66a-45ad-aad4-9b9ad3144f36 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:28.719027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:28.720547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:28.721052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:28.721536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:28.748274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:28.750839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:28.768942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1103/2165] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:29.826295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28576274-6328-4abe-9442-b02815934f80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:29.827309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28576274-6328-4abe-9442-b02815934f80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:29.827826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28576274-6328-4abe-9442-b02815934f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:29.899406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28576274-6328-4abe-9442-b02815934f80 None None] MFA Rules not processed for user `1690be2d7032472391fa96e23e0c1bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:29.901047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28576274-6328-4abe-9442-b02815934f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:30.057615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2166] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:30.083325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:30.087689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:30.177927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:30.226587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74abc7-bd87-429c-94c1-8bb526aa6e28 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:30.235468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:30.236653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:30.237165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:30.237964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:30.255611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:30.259532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:30.287320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1104/2167] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:30.558372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28576274-6328-4abe-9442-b02815934f80 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:30.561281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28576274-6328-4abe-9442-b02815934f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:30.655764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28576274-6328-4abe-9442-b02815934f80 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:30.676798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:30.679727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:30.680233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:30.680678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] PATH_INFO: `/v3/projects/002ef70b05d34cbca6573a917c491150` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:30.687079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=002ef70b05d34cbca6573a917c491150)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:30.691694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:30.851215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002ef70b05d34cbca6573a917c491150', 'request_id': 'req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:30.875184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002ef70b05d34cbca6573a917c491150', 'request_id': 'req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:30.895074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:30.901445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002ef70b05d34cbca6573a917c491150', 'request_id': 'req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:30.937677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] The token cache is being invalidate because project 002ef70b05d34cbca6573a917c491150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:30.938254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:30.941943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:30.947695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2168] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:30 2022] DELETE /identity/v3/projects/002ef70b05d34cbca6573a917c491150 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:30.952790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:30.977197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29f03-077c-48ed-a721-dac35f06d98b service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:31.219004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:31.219728 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:31.220248 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:31.220595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] PATH_INFO: `/v3/users/a83499189b3e4af09e00f4dc842b95dd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:31.221953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83499189b3e4af09e00f4dc842b95dd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:31.224486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:31.318436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83499189b3e4af09e00f4dc842b95dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:31.335190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83499189b3e4af09e00f4dc842b95dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:31.338017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83499189b3e4af09e00f4dc842b95dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:31.338987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83499189b3e4af09e00f4dc842b95dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:31.385413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cad49055-dee2-4213-9e8f-1b3d7f161c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83499189b3e4af09e00f4dc842b95dd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:31.417683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1105/2169] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:30 2022] DELETE /identity/v3/users/a83499189b3e4af09e00f4dc842b95dd => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:32.223774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:32.226108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:32.251157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c32d0-d2b5-4401-a010-7cc85b6ae70e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:32.284632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a162bf-a103-406e-97ff-a1f1024a9455 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:32.285360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a162bf-a103-406e-97ff-a1f1024a9455 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:32.285360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a162bf-a103-406e-97ff-a1f1024a9455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:32.361032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-80a162bf-a103-406e-97ff-a1f1024a9455 None None] MFA Rules not processed for user `45ae88234e984149be8539757efa8733`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:32.362617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-80a162bf-a103-406e-97ff-a1f1024a9455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:32.392402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:32.394691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:32.395016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:32.395305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] PATH_INFO: `/v3/projects/319b0a77605d44699080dcb413848102` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:32.397936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=319b0a77605d44699080dcb413848102)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:32.400115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:32.457108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:32.479580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1106/2170] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:32.501278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57ad9-b270-4d4b-aef7-d619a5c8783a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:32.502269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57ad9-b270-4d4b-aef7-d619a5c8783a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:32.503114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57ad9-b270-4d4b-aef7-d619a5c8783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:32.550193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319b0a77605d44699080dcb413848102', 'request_id': 'req-d8d3109f-6de0-4aed-971d-36ae25df1aee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:32.559766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319b0a77605d44699080dcb413848102', 'request_id': 'req-d8d3109f-6de0-4aed-971d-36ae25df1aee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:32.575666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-17f57ad9-b270-4d4b-aef7-d619a5c8783a None None] MFA Rules not processed for user `1690be2d7032472391fa96e23e0c1bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:32.575666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17f57ad9-b270-4d4b-aef7-d619a5c8783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:32.583421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319b0a77605d44699080dcb413848102', 'request_id': 'req-d8d3109f-6de0-4aed-971d-36ae25df1aee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:32.612576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] The token cache is being invalidate because project 319b0a77605d44699080dcb413848102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:32.613260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:32.615604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8d3109f-6de0-4aed-971d-36ae25df1aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:32.623404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2171] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:32 2022] DELETE /identity/v3/projects/319b0a77605d44699080dcb413848102 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:32.631896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:32.633086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:32.634016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:32.684294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:32.687073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:32.772842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1107/2172] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:32.906420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2173] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:32.909581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:32.910567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:32.911082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:32.962906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:32.964651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:32.980943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1108/2174] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:32.984711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:32.999929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35378cf2-8c5d-41b6-b9f8-377c5278cf59 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.009697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.010353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.010637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.010924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.013832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:23:33.014181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.022613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.057881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2175] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:23:33.058132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:33.072332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3058ba-1026-4ff7-9855-c0681c74cb92 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.080441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.081789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.082391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.082920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.083999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.086813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.127428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1109/2176] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:33 2022] POST /identity/v3/projects => generated 363 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:23:33.134227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:33.147671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a174-731e-4975-942a-1c76a554ee88 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.161692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:33.320323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.321242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.321658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.322181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.323127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.326448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.377896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2177] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:33 2022] POST /identity/v3/users => generated 282 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:33.382804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:33.400028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd17ee7-3421-4dd3-be5f-9a7905c9e654 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.407825 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.408714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.409371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.410112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.411043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.413922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.428742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1110/2178] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:33.432979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:33.446369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701a133-4864-4139-b268-bad9603728e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.452340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddbe43-bedb-4a6a-853d-2452da8dd324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.453377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddbe43-bedb-4a6a-853d-2452da8dd324 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.453820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddbe43-bedb-4a6a-853d-2452da8dd324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.454333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddbe43-bedb-4a6a-853d-2452da8dd324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.455274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddbe43-bedb-4a6a-853d-2452da8dd324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.457112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddbe43-bedb-4a6a-853d-2452da8dd324 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.474706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2179] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:33.479211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:33.499250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445be7a-2e62-4e01-ba92-b74629bdf0c6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.506733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.507637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.508164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.508598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] PATH_INFO: `/v3/projects/11af284622464a6ca9cbc12ab171c6d0/users/92692f98ba5d45268c9dcdc9954cf1a7/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.530364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11af284622464a6ca9cbc12ab171c6d0, user_id=92692f98ba5d45268c9dcdc9954cf1a7, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.533616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.552303 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1111/2180] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:33 2022] PUT /identity/v3/projects/11af284622464a6ca9cbc12ab171c6d0/users/92692f98ba5d45268c9dcdc9954cf1a7/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:33.566332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.566332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.566332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.621891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] MFA Rules not processed for user `92692f98ba5d45268c9dcdc9954cf1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:33.623592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:33.736723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2181] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:33 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:33.743057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:33.906759 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510564-8510-42fe-93cd-b7d18e43d197 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:33.916135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:33.917060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:33.918664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:33.921304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:33.923660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:33.932177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:33.990347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1112/2182] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:33 2022] POST /identity/v3/projects => generated 365 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:33.992231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:34.013591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8f4e4-af52-4cfc-85d2-3e2ce851aa0d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:34.221148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:34.221863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:34.224165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:34.227171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:34.230075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:34.244239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:34.337339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2183] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:33 2022] POST /identity/v3/users => generated 289 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:34.342058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:34.358622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a209d4a-1341-488e-a077-932a2a8cefd2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:34.375186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:34.376381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:34.377135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:34.377793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:34.379977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:34.384986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:34.422404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1113/2184] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:34.424299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:34.450475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:34.472889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d42e00-21ef-425b-b509-87d26e6c5493 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:34.485111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:34.486444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:34.487141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:34.490513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] PATH_INFO: `/v3/projects/bcebeea9afe344c698ea393291bc4df1/users/96b21096e567412ea457f675e25511c7/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:34.561982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcebeea9afe344c698ea393291bc4df1, user_id=96b21096e567412ea457f675e25511c7, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:34.568629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:34.598122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2185] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:34 2022] PUT /identity/v3/projects/bcebeea9afe344c698ea393291bc4df1/users/96b21096e567412ea457f675e25511c7/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:34.601945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:34.617876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8b496-e4c0-494a-bbff-9bab844ae9c8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:34.637346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:34.731622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6b701-724c-497d-9743-5864f7714b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:34.732754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6b701-724c-497d-9743-5864f7714b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:34.733144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6b701-724c-497d-9743-5864f7714b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:34.733703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6b701-724c-497d-9743-5864f7714b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:34.734748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6b701-724c-497d-9743-5864f7714b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:34.737139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6b701-724c-497d-9743-5864f7714b0b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:34.752576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1114/2186] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:34.756557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:34.768385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0041a5a-b14d-4c14-9a3d-60af8f25d8fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:34.774883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:34.776799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:34.777968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:34.778441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] PATH_INFO: `/v3/projects/bcebeea9afe344c698ea393291bc4df1/users/96b21096e567412ea457f675e25511c7/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:34.783094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcebeea9afe344c698ea393291bc4df1, user_id=96b21096e567412ea457f675e25511c7, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:34.787000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:34.835910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2187] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:34 2022] PUT /identity/v3/projects/bcebeea9afe344c698ea393291bc4df1/users/96b21096e567412ea457f675e25511c7/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:34.851109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:34.851777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:34.852252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:34.921792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] MFA Rules not processed for user `96b21096e567412ea457f675e25511c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:34.923444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:35.059156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1115/2188] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:35.087751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:35.089207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:35.233945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4055e-39a5-41e0-b656-a60279d87f81 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:35.240011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d7383-bf71-4305-b718-1449aefb2c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:35.241858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d7383-bf71-4305-b718-1449aefb2c44 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:35.242621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d7383-bf71-4305-b718-1449aefb2c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:35.242621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d7383-bf71-4305-b718-1449aefb2c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:35.254566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d7383-bf71-4305-b718-1449aefb2c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:35.256123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d7383-bf71-4305-b718-1449aefb2c44 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:35.270429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2189] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:35.849083 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:35.928584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fe4c3-7b3e-4bb7-b504-2988fc8a2314 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:35.934784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:35.935497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:35.935797 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:35.936079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] PATH_INFO: `/v3/users/92692f98ba5d45268c9dcdc9954cf1a7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:35.937456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=92692f98ba5d45268c9dcdc9954cf1a7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:35.939523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:35.994168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:36.007670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92692f98ba5d45268c9dcdc9954cf1a7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:36.017952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92692f98ba5d45268c9dcdc9954cf1a7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:36.018618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92692f98ba5d45268c9dcdc9954cf1a7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:36.019239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92692f98ba5d45268c9dcdc9954cf1a7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:36.035884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92692f98ba5d45268c9dcdc9954cf1a7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:36.053353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1116/2190] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:35 2022] DELETE /identity/v3/users/92692f98ba5d45268c9dcdc9954cf1a7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:36.059371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:36.059954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:36.060499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:36.099861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:36.101872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:36.228985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2191] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:36.297184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:36.298796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:36.465816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c964f7-4d33-40c9-832f-65f7e0f986ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:36.473720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:36.474987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:36.475840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:36.476600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:36.500833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:36.502927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:36.525376 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1117/2192] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:37.053792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:37.055580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:37.079635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85fd65-e990-495b-84b2-ebd521884ed3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:37.110367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:37.112976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:37.113478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:37.114022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] PATH_INFO: `/v3/projects/11af284622464a6ca9cbc12ab171c6d0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:37.121629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=11af284622464a6ca9cbc12ab171c6d0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:37.128156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:37.220865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11af284622464a6ca9cbc12ab171c6d0', 'request_id': 'req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.228166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11af284622464a6ca9cbc12ab171c6d0', 'request_id': 'req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.234977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11af284622464a6ca9cbc12ab171c6d0', 'request_id': 'req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.237997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:37.258231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] The token cache is being invalidate because project 11af284622464a6ca9cbc12ab171c6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:37.258711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.259779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.272839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2193] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:37 2022] DELETE /identity/v3/projects/11af284622464a6ca9cbc12ab171c6d0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:37.276915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:37.304583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669d859-c5d0-4102-8f4b-26142236d946 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:37.511563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:37.512354 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:37.512647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:37.512929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] PATH_INFO: `/v3/users/96b21096e567412ea457f675e25511c7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:37.514340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b21096e567412ea457f675e25511c7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:37.516408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:37.620216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b21096e567412ea457f675e25511c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.634895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b21096e567412ea457f675e25511c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.636017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b21096e567412ea457f675e25511c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.637415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b21096e567412ea457f675e25511c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.647156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:37.665810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb1c21e2-8fa0-43c6-a5a2-0b98cccf0a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b21096e567412ea457f675e25511c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:37.685525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1118/2194] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:37 2022] DELETE /identity/v3/users/96b21096e567412ea457f675e25511c7 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:38.394135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:38.396732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:38.432770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98172127-abd3-4c81-b97a-ac8c0b7f6f0e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:38.571796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:38.573667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:38.574657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:38.575444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] PATH_INFO: `/v3/projects/bcebeea9afe344c698ea393291bc4df1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:38.580227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcebeea9afe344c698ea393291bc4df1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:38.587035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:38.712948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcebeea9afe344c698ea393291bc4df1', 'request_id': 'req-9495235d-491f-4ee9-a711-4ed4391521ec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:38.725739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcebeea9afe344c698ea393291bc4df1', 'request_id': 'req-9495235d-491f-4ee9-a711-4ed4391521ec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:38.752052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcebeea9afe344c698ea393291bc4df1', 'request_id': 'req-9495235d-491f-4ee9-a711-4ed4391521ec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:38.794133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] The token cache is being invalidate because project bcebeea9afe344c698ea393291bc4df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:38.797739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:38.800826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9495235d-491f-4ee9-a711-4ed4391521ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:38.804671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2195] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:38 2022] DELETE /identity/v3/projects/bcebeea9afe344c698ea393291bc4df1 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:38.808906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:38.809714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:38.809714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:38.834360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:38.836751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:38.975431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1119/2196] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:38.984268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509199-ca37-4319-866d-606ebaee260c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:38.985096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509199-ca37-4319-866d-606ebaee260c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:38.985761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509199-ca37-4319-866d-606ebaee260c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.029709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49509199-ca37-4319-866d-606ebaee260c None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:39.032679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49509199-ca37-4319-866d-606ebaee260c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:39.045178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2197] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:39.049753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.061146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb9625-5a42-4128-b1c1-7fd28ebff6c7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.066408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.067065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.067383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.067674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.068432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:23:39.068746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.070544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.094406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1120/2198] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:23:39.100657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509199-ca37-4319-866d-606ebaee260c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.116479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509199-ca37-4319-866d-606ebaee260c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.123436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.124290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.124658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.124997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.125758 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.128745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.172653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2199] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:39 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:39.179733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.210877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c7d8-b26c-4caa-9ce3-2868b9b8ab40 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.303918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.305213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.305716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.306337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.307694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.310440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.358207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1121/2200] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:39 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:23:39.360434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.379338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b6c85-83e6-442e-831f-a999ab42bc60 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.388641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.389347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.390629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.390629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.391656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.394030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.414188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2201] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:39.418350 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.436997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55870941-07d1-414c-ab03-0db22067ccfc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.445913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cff12-780d-43bb-9e0f-831656fd5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.448295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cff12-780d-43bb-9e0f-831656fd5234 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.449276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cff12-780d-43bb-9e0f-831656fd5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.450003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cff12-780d-43bb-9e0f-831656fd5234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.453478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cff12-780d-43bb-9e0f-831656fd5234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.457027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cff12-780d-43bb-9e0f-831656fd5234 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.476770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1122/2202] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:39.481464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.495621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde40c92-d34f-43b5-8820-0b8b5db44f79 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.502259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.503573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.504201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.504765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] PATH_INFO: `/v3/projects/48c58a138afd4b7cb6bf3f7c1dbb6860/users/d6ac595c864c4db2b33c0b812154712e/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.525129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c58a138afd4b7cb6bf3f7c1dbb6860, user_id=d6ac595c864c4db2b33c0b812154712e, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.529089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.549828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2203] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:39 2022] PUT /identity/v3/projects/48c58a138afd4b7cb6bf3f7c1dbb6860/users/d6ac595c864c4db2b33c0b812154712e/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:39.561049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.561356 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.561638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.614514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] MFA Rules not processed for user `d6ac595c864c4db2b33c0b812154712e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:39.616401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:39.659230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:39.728833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1123/2204] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:39.732158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.781686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:39.890382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0579d-025c-4c16-9284-cd346f782669 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:39.897389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:39.898212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:39.898545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:39.898938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:39.899699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:39.903038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:39.967688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:39.973092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2205] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:39 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:39.977532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655fb28-5fe2-424f-b92c-415b1d682819 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:40.066552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:40.067258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:40.067659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:40.068005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:40.068659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:40.072372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:40.119070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:40.132489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1124/2206] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:39 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:40.137124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f29f-b568-4ad2-8a99-0d5c8fae2b58 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:40.145002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:40.145575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:40.145871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:40.146157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:40.146969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:40.149334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:40.178178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:40.183752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2207] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:40.210591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee624-274c-4d91-aac2-59158cf42c61 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:40.219571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:40.220830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:40.221339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:40.221829 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] PATH_INFO: `/v3/projects/926311d46c1b44ae8e7970e8ce9f01c8/users/13a436799414484287dbc4443f9ed7e6/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:40.256207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926311d46c1b44ae8e7970e8ce9f01c8, user_id=13a436799414484287dbc4443f9ed7e6, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:40.264819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:40.317475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1125/2208] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:40 2022] PUT /identity/v3/projects/926311d46c1b44ae8e7970e8ce9f01c8/users/13a436799414484287dbc4443f9ed7e6/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:40.321538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:40.339261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f101e09-6578-49fe-8f53-5e4290a661d0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:40.423332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd9fc4-9bd4-486a-8093-8e5eca7b7752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:40.425460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9fc4-9bd4-486a-8093-8e5eca7b7752 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:40.425460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9fc4-9bd4-486a-8093-8e5eca7b7752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:40.425460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9fc4-9bd4-486a-8093-8e5eca7b7752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:40.426965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd9fc4-9bd4-486a-8093-8e5eca7b7752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:40.430118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd9fc4-9bd4-486a-8093-8e5eca7b7752 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:40.451495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2209] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:23:40.456456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:40.476709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18af2c-7e06-46b7-a519-edcdc3b37aa5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:40.486686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:40.487471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:40.487789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:40.488076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] PATH_INFO: `/v3/projects/926311d46c1b44ae8e7970e8ce9f01c8/users/13a436799414484287dbc4443f9ed7e6/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:40.490258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926311d46c1b44ae8e7970e8ce9f01c8, user_id=13a436799414484287dbc4443f9ed7e6, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:40.493065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:40.507496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1126/2210] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:40 2022] PUT /identity/v3/projects/926311d46c1b44ae8e7970e8ce9f01c8/users/13a436799414484287dbc4443f9ed7e6/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:40.518425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:40.518925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:40.519376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:40.581742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] MFA Rules not processed for user `13a436799414484287dbc4443f9ed7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:40.584491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:40.773994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2211] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:40.789439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:40.813495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:41.004528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a83c-26a4-49db-903f-8921beb5b246 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:41.014489 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732bff6-a283-4e70-b7c8-66dfb0f05ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:41.015672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732bff6-a283-4e70-b7c8-66dfb0f05ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:41.016189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732bff6-a283-4e70-b7c8-66dfb0f05ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:41.016767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732bff6-a283-4e70-b7c8-66dfb0f05ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:41.038998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732bff6-a283-4e70-b7c8-66dfb0f05ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:41.042355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732bff6-a283-4e70-b7c8-66dfb0f05ec6 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:41.065400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1127/2212] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:41.099074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:41.135675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:41.269365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f96a-9c46-4563-98ec-5b44bc19947e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:41.280178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:41.281462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:41.282327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:41.282979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] PATH_INFO: `/v3/users/d6ac595c864c4db2b33c0b812154712e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:41.285867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ac595c864c4db2b33c0b812154712e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:41.291036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:41.365698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ac595c864c4db2b33c0b812154712e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:41.366469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ac595c864c4db2b33c0b812154712e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:41.381032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ac595c864c4db2b33c0b812154712e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:41.381866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ac595c864c4db2b33c0b812154712e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:41.406879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ac595c864c4db2b33c0b812154712e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:41.437533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2213] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:41 2022] DELETE /identity/v3/users/d6ac595c864c4db2b33c0b812154712e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:41.443604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:41.444231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:41.444807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:41.493061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:41.503113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:41.593077 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:41.726411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1128/2214] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:41.742068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:41.743288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:41.894176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def5099-26c3-4ce6-9d93-7851eb4635ff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:41.900723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:41.901432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:41.901789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:41.902171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:41.919164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:41.922403 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:41.944406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2215] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:43.535978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:43.537452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:43.565792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023725e9-3bd1-49ab-8f00-5d5acf6795c9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:43.580709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:43.582261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:43.582550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:43.582836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] PATH_INFO: `/v3/projects/48c58a138afd4b7cb6bf3f7c1dbb6860` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:43.584377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c58a138afd4b7cb6bf3f7c1dbb6860)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:43.586442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:43.677535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c58a138afd4b7cb6bf3f7c1dbb6860', 'request_id': 'req-cf778c07-7b2f-4899-ab20-a38c80e51f76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:43.690004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c58a138afd4b7cb6bf3f7c1dbb6860', 'request_id': 'req-cf778c07-7b2f-4899-ab20-a38c80e51f76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:43.707392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c58a138afd4b7cb6bf3f7c1dbb6860', 'request_id': 'req-cf778c07-7b2f-4899-ab20-a38c80e51f76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:43.726067 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] The token cache is being invalidate because project 48c58a138afd4b7cb6bf3f7c1dbb6860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:43.726445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:43.727276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:43.729249 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1129/2216] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:43 2022] DELETE /identity/v3/projects/48c58a138afd4b7cb6bf3f7c1dbb6860 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:43.734710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:43.766302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb5473-662f-4394-9572-5a12cb0a9a8a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:43.856242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:43.928693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:43.930313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:43.930561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:43.930970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] PATH_INFO: `/v3/users/13a436799414484287dbc4443f9ed7e6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:43.934265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a436799414484287dbc4443f9ed7e6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:43.936876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:44.020264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a436799414484287dbc4443f9ed7e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:44.021508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a436799414484287dbc4443f9ed7e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:44.033141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a436799414484287dbc4443f9ed7e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:44.033927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a436799414484287dbc4443f9ed7e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:44.059859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0604c169-223b-4bd9-a80d-7fec34a026e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a436799414484287dbc4443f9ed7e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:44.088606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2217] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:43 2022] DELETE /identity/v3/users/13a436799414484287dbc4443f9ed7e6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:44.696358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:44.710846 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:44.752977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778c07-7b2f-4899-ab20-a38c80e51f76 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:44.927150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:44.928280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:44.928749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:44.929245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] PATH_INFO: `/v3/projects/926311d46c1b44ae8e7970e8ce9f01c8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:44.931695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926311d46c1b44ae8e7970e8ce9f01c8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:44.937287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.063170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926311d46c1b44ae8e7970e8ce9f01c8', 'request_id': 'req-17e97e7a-3309-42e7-baf1-4490879be133'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:45.070559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:45.094229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926311d46c1b44ae8e7970e8ce9f01c8', 'request_id': 'req-17e97e7a-3309-42e7-baf1-4490879be133'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:45.106263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926311d46c1b44ae8e7970e8ce9f01c8', 'request_id': 'req-17e97e7a-3309-42e7-baf1-4490879be133'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:45.137942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] The token cache is being invalidate because project 926311d46c1b44ae8e7970e8ce9f01c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:45.138449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:45.139858 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-17e97e7a-3309-42e7-baf1-4490879be133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:45.146374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1130/2218] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:44 2022] DELETE /identity/v3/projects/926311d46c1b44ae8e7970e8ce9f01c8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:45.149448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.150982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.152788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.224364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:45.225635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:45.448979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2219] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:45.459410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.460060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.460474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.508760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:45.510114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:45.528362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1131/2220] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:45.531088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:45.548155 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090a76-915c-4445-92cf-72d0529635b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:45.559695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:45.560075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.560370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.560659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.561597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:23:45.561918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:45.564343 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.597248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2221] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:23:45.603522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:45.616550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f8c45-395f-468d-9042-61a8f4b0d9ac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:45.631171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:45.631923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.632213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.632503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.633276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:45.636235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.679566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1132/2222] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:45 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:45.680672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:45.700142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8323-590c-41ef-b0e2-9189335d9d47 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:45.759118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:45.785726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:45.786466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.786832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.787133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.787862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:45.789893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.826100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2223] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:45 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:23:45.831906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:45.844675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dbe3d-3198-4762-924d-28567b63a7b7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:45.850355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:45.851006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.851296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.851608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.852515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:45.854166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.866605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1133/2224] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:45.871600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:45.882621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c1a4e-00f6-4995-9a06-de34cb078188 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:45.887863 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab5917-67e8-4591-b9ec-073e9b7b8180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:45.888507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab5917-67e8-4591-b9ec-073e9b7b8180 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.888792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab5917-67e8-4591-b9ec-073e9b7b8180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.889066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab5917-67e8-4591-b9ec-073e9b7b8180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.889767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab5917-67e8-4591-b9ec-073e9b7b8180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:45.891426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab5917-67e8-4591-b9ec-073e9b7b8180 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.904287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2225] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:23:45.908590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:45.922421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff33b6-6101-4bb7-a1f9-2262c7a163a3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:45.939297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:45.939954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.940244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.940527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] PATH_INFO: `/v3/projects/578e96ed553744f4bba43fada85cc38a/users/753545c569944bb4b63d4acb67eb505f/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:45.958663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578e96ed553744f4bba43fada85cc38a, user_id=753545c569944bb4b63d4acb67eb505f, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:45.961474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:45.978926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1134/2226] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:45 2022] PUT /identity/v3/projects/578e96ed553744f4bba43fada85cc38a/users/753545c569944bb4b63d4acb67eb505f/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:45.993966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:45.994498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:45.995089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.033549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] MFA Rules not processed for user `753545c569944bb4b63d4acb67eb505f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:46.035061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:46.129399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2227] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:46.134357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.224611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a60c44-34e8-45da-b7c7-2995ef825499 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.230584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.231314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.231694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.231989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.232682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.234697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:46.271993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1135/2228] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:46 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:46.276204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.288444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa602e0-2fad-45f6-a7bf-628897f19227 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.353479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.354142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.355327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.355327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.356142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.358163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:46.388832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2229] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:46 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:46.392802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.404488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5d0d9-0f93-45d8-a33b-f4bd8aabeb76 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.410259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.410913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.411203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.411536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.412281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.413968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:46.427653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1136/2230] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:46.432147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.444083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de27a62-8f15-41a7-a9c8-43350bf065e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.449286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.449992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.450338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.450627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] PATH_INFO: `/v3/projects/c5bee50dc1a54c5e9cdfabe801cb0e3c/users/1e8b941719ba412ab993c030da1cfaf0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.468815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bee50dc1a54c5e9cdfabe801cb0e3c, user_id=1e8b941719ba412ab993c030da1cfaf0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.471485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:46.492820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2231] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:46 2022] PUT /identity/v3/projects/c5bee50dc1a54c5e9cdfabe801cb0e3c/users/1e8b941719ba412ab993c030da1cfaf0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:46.498130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.513636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd94692-91f3-4795-a41d-b2d64e448de4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.572642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:46.585496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d9b03-03a6-4cc1-8080-8e71f39aeabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.586139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d9b03-03a6-4cc1-8080-8e71f39aeabc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.586428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d9b03-03a6-4cc1-8080-8e71f39aeabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.586720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d9b03-03a6-4cc1-8080-8e71f39aeabc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.587456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d9b03-03a6-4cc1-8080-8e71f39aeabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.589297 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d9b03-03a6-4cc1-8080-8e71f39aeabc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:46.601911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1137/2232] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:23:46.606978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.624622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912daff-9a95-4468-b239-7375ad519893 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.632396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.633479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.633984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.634490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] PATH_INFO: `/v3/projects/c5bee50dc1a54c5e9cdfabe801cb0e3c/users/1e8b941719ba412ab993c030da1cfaf0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.638361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bee50dc1a54c5e9cdfabe801cb0e3c, user_id=1e8b941719ba412ab993c030da1cfaf0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.643746 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:46.663324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2233] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:46 2022] PUT /identity/v3/projects/c5bee50dc1a54c5e9cdfabe801cb0e3c/users/1e8b941719ba412ab993c030da1cfaf0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:46.675840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.676137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.676423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.707692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] MFA Rules not processed for user `1e8b941719ba412ab993c030da1cfaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:46.708939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:46.833170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1138/2234] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:46.846816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:46.850326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:46.858294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:46.963480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd699-1296-4a15-8284-b979db2aad44 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:46.972752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:46.973858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:46.974351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:46.974855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:46.987830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:46.989655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:47.006556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2235] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:47.153006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:47.163143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d3d62-fc86-433b-8ee6-cc7f06efc0b9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:47.168563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4900b-28af-4374-a439-e455b9296231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:47.169305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4900b-28af-4374-a439-e455b9296231 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:47.169653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4900b-28af-4374-a439-e455b9296231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:47.169995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4900b-28af-4374-a439-e455b9296231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:47.254909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4900b-28af-4374-a439-e455b9296231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:47.256357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4900b-28af-4374-a439-e455b9296231 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:47.265980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1139/2236] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:49.978171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:50.083829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa301ee-5633-41d7-9e93-c1beba8be4b3 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:50.090413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:50.091374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:50.091750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:50.092094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] PATH_INFO: `/v3/users/753545c569944bb4b63d4acb67eb505f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:50.093373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=753545c569944bb4b63d4acb67eb505f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:50.095472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:50.179550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753545c569944bb4b63d4acb67eb505f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:50.181034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753545c569944bb4b63d4acb67eb505f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:50.195654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753545c569944bb4b63d4acb67eb505f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:50.200748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753545c569944bb4b63d4acb67eb505f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:50.217927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753545c569944bb4b63d4acb67eb505f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:50.234548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2237] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:49 2022] DELETE /identity/v3/users/753545c569944bb4b63d4acb67eb505f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:50.239194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:50.239836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:50.239836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:50.268371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:50.269511 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:50.358213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1140/2238] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:50.370594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:50.373218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:50.431833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:50.481215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb592a-fd41-4890-9013-b238a9a59f76 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:50.488375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:50.489529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:50.490053 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:50.490479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:50.509278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:50.511650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:50.530031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2239] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:51.331048 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:51.334421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:51.361641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2b52e-f30f-425a-a66b-11dbb449eaa8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:51.371481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:51.372997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:51.373900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:51.374607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] PATH_INFO: `/v3/projects/578e96ed553744f4bba43fada85cc38a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:51.377246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=578e96ed553744f4bba43fada85cc38a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:51.381145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:51.395675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:51.465192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578e96ed553744f4bba43fada85cc38a', 'request_id': 'req-128befdd-9742-4736-a725-82b01eb66b4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.476129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578e96ed553744f4bba43fada85cc38a', 'request_id': 'req-128befdd-9742-4736-a725-82b01eb66b4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.484049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578e96ed553744f4bba43fada85cc38a', 'request_id': 'req-128befdd-9742-4736-a725-82b01eb66b4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.504603 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] The token cache is being invalidate because project 578e96ed553744f4bba43fada85cc38a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:51.506472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.507289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-128befdd-9742-4736-a725-82b01eb66b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.511131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1141/2240] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:51 2022] DELETE /identity/v3/projects/578e96ed553744f4bba43fada85cc38a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:51.517051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:51.528213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57aa1af-7dd7-4f83-b56d-d7c09440dfcb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:51.624843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:51.625876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:51.626881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:51.626881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] PATH_INFO: `/v3/users/1e8b941719ba412ab993c030da1cfaf0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:51.630384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e8b941719ba412ab993c030da1cfaf0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:51.634500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:51.720454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e8b941719ba412ab993c030da1cfaf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.721518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e8b941719ba412ab993c030da1cfaf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.735812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e8b941719ba412ab993c030da1cfaf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.736976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e8b941719ba412ab993c030da1cfaf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.764842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e8b941719ba412ab993c030da1cfaf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:51.790767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2241] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:51 2022] DELETE /identity/v3/users/1e8b941719ba412ab993c030da1cfaf0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:52.302471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:52.302887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:52.303109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:52.343933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:52.345859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:52.359564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:52.363128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:52.502535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:52.528866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1142/2242] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:23:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:52.536793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0937885-2b3f-4582-b99e-3c25e1ac0994 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:52.546973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:52.547709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:52.548116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:52.548531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] PATH_INFO: `/v3/projects/c5bee50dc1a54c5e9cdfabe801cb0e3c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:52.549797 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bee50dc1a54c5e9cdfabe801cb0e3c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:52.552607 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:52.583301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:52.584804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:52.663987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bee50dc1a54c5e9cdfabe801cb0e3c', 'request_id': 'req-06ac59f6-07c1-4fd9-a794-ebf986507473'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:52.692194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bee50dc1a54c5e9cdfabe801cb0e3c', 'request_id': 'req-06ac59f6-07c1-4fd9-a794-ebf986507473'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:52.692194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bee50dc1a54c5e9cdfabe801cb0e3c', 'request_id': 'req-06ac59f6-07c1-4fd9-a794-ebf986507473'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:52.717754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] The token cache is being invalidate because project c5bee50dc1a54c5e9cdfabe801cb0e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:23:52.718378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:52.719699 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06ac59f6-07c1-4fd9-a794-ebf986507473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:52.725844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2243] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:23:52 2022] DELETE /identity/v3/projects/c5bee50dc1a54c5e9cdfabe801cb0e3c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:52.728931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dc637-208f-4ee1-9a69-d0047ec72ffb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:52.729994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dc637-208f-4ee1-9a69-d0047ec72ffb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:52.730298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dc637-208f-4ee1-9a69-d0047ec72ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:52.731182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6c34d-c5e0-477e-bb74-29e508f1f81f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:52.767790 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-586dc637-208f-4ee1-9a69-d0047ec72ffb None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:52.770073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-586dc637-208f-4ee1-9a69-d0047ec72ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:52.832538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:52.879618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:52.895650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:52.896070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:52.896362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:52.898804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:52.936921 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2244] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:52.946757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:52.947089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:52.947441 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:52.972428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:52.973679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:52.983265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2245] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:52.987531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:52.999452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.001170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22b937-0fb0-4260-b7ff-49ad99cd68ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:53.001343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.008498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:53.009706 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.010258 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.011393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.012977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:23:53.013668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1143/2246] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:23:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 05:23:53.020089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.024396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.054359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2247] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:23:53.059860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:53.075839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3414d-8c0d-4424-aa5f-d0c27bc33eac service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:53.081551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:53.082965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.083562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.084027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.085118 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.087661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.135910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1144/2248] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:53 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:53.140508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:53.155187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a06df-0796-4c18-a824-65b43412ab49 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:53.246384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:53.247159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.247866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.248018 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.249204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.254755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.267930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:53.299404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2249] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:53 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:53.301338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:53.319474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05f2ed-297b-421d-ba44-dd515e182c6f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:53.329690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:53.332001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.332531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.333239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.335758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.342756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.363614 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1145/2250] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:53.367261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:53.390266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e92a3-f3b3-43da-8ef6-4d1b096c7d6e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:53.399834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a886283-b65e-495a-8450-ef894cc82225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:53.400408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a886283-b65e-495a-8450-ef894cc82225 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.400645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a886283-b65e-495a-8450-ef894cc82225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.401455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a886283-b65e-495a-8450-ef894cc82225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.402349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a886283-b65e-495a-8450-ef894cc82225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.404099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a886283-b65e-495a-8450-ef894cc82225 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.430163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2251] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:53.435739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:53.455266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505acbed-1387-4b9a-a250-1294658add08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:53.463934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:53.465068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.465548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.466004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] PATH_INFO: `/v3/projects/d8b159e657d843cb82aeddc3ac276128/users/ab248918edad4baeac0e2e7a3ae25bd4/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.497742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b159e657d843cb82aeddc3ac276128, user_id=ab248918edad4baeac0e2e7a3ae25bd4, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:53.503078 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:53.528516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1146/2252] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:53 2022] PUT /identity/v3/projects/d8b159e657d843cb82aeddc3ac276128/users/ab248918edad4baeac0e2e7a3ae25bd4/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:53.539227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14d4b4-c8eb-4d6e-a9cc-97cd3dd4ea5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.539773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14d4b4-c8eb-4d6e-a9cc-97cd3dd4ea5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.539872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14d4b4-c8eb-4d6e-a9cc-97cd3dd4ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.574201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b14d4b4-c8eb-4d6e-a9cc-97cd3dd4ea5b None None] MFA Rules not processed for user `ab248918edad4baeac0e2e7a3ae25bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:53.575813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b14d4b4-c8eb-4d6e-a9cc-97cd3dd4ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:53.678625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2253] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:53.683389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:53.897737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:53.898109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:53.898368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:53.946322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:53.953023 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:54.016014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a3d09-c7c5-4e88-b938-cfe561fee8b0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.039849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.040745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.041360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.042597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.042857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.045231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.100066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1147/2254] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:53 2022] POST /identity/v3/projects => generated 379 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:23:54.100360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2255] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 05:23:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:54.101271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.119263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.121906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:54.125805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:54.264816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0717-6ce0-4bed-addc-0cfcbbbf674d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.266952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de7ac5-c120-4219-94c2-13dc543c7713 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.273196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.274057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.274502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.274883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.275742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.278778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.297261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:54.339393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1148/2256] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:54 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:54.342706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.364571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2c0fe-3814-493d-ab3d-9d960c2b8fb3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.375403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.376778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.377595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.378494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.379778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.382275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.399832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1149/2257] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:54.407231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.419639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.420187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.420476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.420756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.426431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3db9b-7326-4a86-82f3-d9fdec44d11c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.433515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.434175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.434462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.434743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.435499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.437149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.453330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1150/2258] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:23:54.458853 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.482551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f5420-d0fb-48f9-8c24-bb118e4827e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.493580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097c927-b2f2-47be-badb-e96c73d7275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.495216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097c927-b2f2-47be-badb-e96c73d7275f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.495736 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097c927-b2f2-47be-badb-e96c73d7275f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.496186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097c927-b2f2-47be-badb-e96c73d7275f admin admin] PATH_INFO: `/v3/projects/da46cb1cbf8b4a969fa0b13cd913b666/users/bd1e7d7f7a0f4b1ea89b4747d9824867/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.525372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097c927-b2f2-47be-badb-e96c73d7275f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da46cb1cbf8b4a969fa0b13cd913b666, user_id=bd1e7d7f7a0f4b1ea89b4747d9824867, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.530546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097c927-b2f2-47be-badb-e96c73d7275f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.546498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.550958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.555409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1151/2259] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:54 2022] PUT /identity/v3/projects/da46cb1cbf8b4a969fa0b13cd913b666/users/bd1e7d7f7a0f4b1ea89b4747d9824867/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:54.572414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.572798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.573381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.585078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2260] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:23:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:23:54.612947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] MFA Rules not processed for user `bd1e7d7f7a0f4b1ea89b4747d9824867`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:54.614346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:54.731425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1152/2261] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:54.735502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.768782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a076c94-0382-4886-bb79-2628cd3a592c service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.850058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5115b73-7252-43da-843d-e99a8c3b68bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:54.850897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5115b73-7252-43da-843d-e99a8c3b68bd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.851176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5115b73-7252-43da-843d-e99a8c3b68bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.851520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5115b73-7252-43da-843d-e99a8c3b68bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.852233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5115b73-7252-43da-843d-e99a8c3b68bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:54.854739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5115b73-7252-43da-843d-e99a8c3b68bd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:54.894139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2262] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:23:54 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:54.898079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:54.917159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3dd94-82fc-4ad6-bf90-9cd88239defd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:54.930069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:54.931200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:54.933148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:54.959432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:54.961240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:54.997189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:55.032994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.033380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:55.033648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.034191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.034620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.035323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.037390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.067891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2263] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:55.081090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.099169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1153/2264] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:23:54 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:23:55.100219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.120299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1758f34-c820-407c-bf8c-8550ac3096ad admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.141527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.142831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.144287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.144287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.145072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.146786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.165638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1154/2265] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:55.171585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.195098 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fe17-efbd-4bcd-b815-5cf7edb48dbb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.204919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.205774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.206667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.206667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] PATH_INFO: `/v3/projects/4a2ce2657d6c41129fb16427f40ee56e/users/419132e29a614919a0ba9b14e54a60c0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.221335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b76228-b31c-46a3-8987-015cf04600fa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.227401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2ce2657d6c41129fb16427f40ee56e, user_id=419132e29a614919a0ba9b14e54a60c0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.227595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.228311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.228600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.228910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.231377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.243425 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.248679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.253194 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1155/2266] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:55 2022] PUT /identity/v3/projects/4a2ce2657d6c41129fb16427f40ee56e/users/419132e29a614919a0ba9b14e54a60c0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:23:55.257441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.263905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2267] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:23:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:23:55.371551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15adb5-7de2-42a6-905d-ea4b3ceb2990 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.377257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddf368-cf87-45f9-bfa2-2a4f50f3379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.377930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddf368-cf87-45f9-bfa2-2a4f50f3379c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.378267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddf368-cf87-45f9-bfa2-2a4f50f3379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.378548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddf368-cf87-45f9-bfa2-2a4f50f3379c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.379298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddf368-cf87-45f9-bfa2-2a4f50f3379c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.381178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddf368-cf87-45f9-bfa2-2a4f50f3379c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.399409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1156/2268] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:23:55.402627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.423629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61389e-4096-4fad-857a-036ace38c538 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.430318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.431766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.432910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.433603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] PATH_INFO: `/v3/projects/4a2ce2657d6c41129fb16427f40ee56e/users/419132e29a614919a0ba9b14e54a60c0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.438570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2ce2657d6c41129fb16427f40ee56e, user_id=419132e29a614919a0ba9b14e54a60c0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.444306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.466019 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2269] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:23:55 2022] PUT /identity/v3/projects/4a2ce2657d6c41129fb16427f40ee56e/users/419132e29a614919a0ba9b14e54a60c0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:55.486116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d8a45-1a8d-4abb-8efb-9c266270709a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.486764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d8a45-1a8d-4abb-8efb-9c266270709a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.487115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d8a45-1a8d-4abb-8efb-9c266270709a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.546767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1a1d8a45-1a8d-4abb-8efb-9c266270709a None None] MFA Rules not processed for user `419132e29a614919a0ba9b14e54a60c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:55.549710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1a1d8a45-1a8d-4abb-8efb-9c266270709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:55.668165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1157/2270] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:23:55.686140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.688828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:55.777557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633550c0-ad88-4d9f-a239-db98db71b799 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.781557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.782207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.782782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.782782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.805166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:55.806839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:55.818683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2271] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:55.868774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.869289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.870245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.871244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:55.873029 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:55.886248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e69010-34fc-4b0c-bd30-9bb1cee3cc80 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:55.890320 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:55.890950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:55.891242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:55.891604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:55.901732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:55.904260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:55.929951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:56.005713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1158/2272] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 05:23:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:56.019936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:56.023771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:56.028952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:56.052101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2273] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:23:56.125111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd5a80-2f55-4331-8e0d-ab400eb5e621 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:56.134681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5833ea-4bd3-44a7-8618-2fc8247aa4d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:56.136596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5833ea-4bd3-44a7-8618-2fc8247aa4d7 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:56.136596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5833ea-4bd3-44a7-8618-2fc8247aa4d7 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:56.136943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5833ea-4bd3-44a7-8618-2fc8247aa4d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:56.154443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5833ea-4bd3-44a7-8618-2fc8247aa4d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:56.157369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5833ea-4bd3-44a7-8618-2fc8247aa4d7 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:56.179247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1159/2274] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:23:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 05:23:58.905774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:58.908546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:58.921043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:59.048333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5990b4-5a04-4588-a653-b9df08719675 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:59.054524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:59.055290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:59.055836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:59.056457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] PATH_INFO: `/v3/users/073b335a6ba0400e83608e8a862904ee` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:59.058787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] RBAC: Authorizing `identity:delete_user(user_id=073b335a6ba0400e83608e8a862904ee)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:59.061573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:59.157710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073b335a6ba0400e83608e8a862904ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.158497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073b335a6ba0400e83608e8a862904ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.181371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073b335a6ba0400e83608e8a862904ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.184304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073b335a6ba0400e83608e8a862904ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.213693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073b335a6ba0400e83608e8a862904ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.257677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2275] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:58 2022] DELETE /identity/v3/users/073b335a6ba0400e83608e8a862904ee => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:59.261036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:59.261615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:59.261992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:59.302934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:59.305147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:59.454465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1160/2276] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:23:59.467562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:59.469973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:59.596966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:23:59.599612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:23:59.623934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefd209-6d5d-4b9b-a5d6-4a8ab550ef0b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:59.656973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:59.659509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:59.660548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:59.661172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] PATH_INFO: `/v3/users/ab248918edad4baeac0e2e7a3ae25bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:59.664683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab248918edad4baeac0e2e7a3ae25bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:59.669527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:59.717192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ec58a-f668-42ef-b2c7-2f156ed8937a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:23:59.723334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f519-a6bd-43ee-bf78-bb2794a68b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:23:59.726000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f519-a6bd-43ee-bf78-bb2794a68b45 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:59.726000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f519-a6bd-43ee-bf78-bb2794a68b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:59.726382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f519-a6bd-43ee-bf78-bb2794a68b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:59.749448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f519-a6bd-43ee-bf78-bb2794a68b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:23:59.752283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f519-a6bd-43ee-bf78-bb2794a68b45 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:23:59.771421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:23:59.781522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab248918edad4baeac0e2e7a3ae25bd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.788878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:23:59.800434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab248918edad4baeac0e2e7a3ae25bd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.801677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab248918edad4baeac0e2e7a3ae25bd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.802408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab248918edad4baeac0e2e7a3ae25bd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.840302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab248918edad4baeac0e2e7a3ae25bd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:23:59.868151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1161/2278] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:23:59 2022] DELETE /identity/v3/users/ab248918edad4baeac0e2e7a3ae25bd4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:23:59.878153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:23:59.878584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:23:59.879414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:23:59.933007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:23:59.953756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:00.008986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:00.181462 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2279] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:24:00.231892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:00.234404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:00.361885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:00.363575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:00.378206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb012bc-68bc-414b-8906-db5e3aadf304 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:00.385005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:00.386271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:00.386271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:00.386271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] PATH_INFO: `/v3/projects/4804e980ecc84e12a90f01bca1acee0a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:00.389059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4804e980ecc84e12a90f01bca1acee0a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:00.402555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:00.410120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939ac8-6f5c-4744-bbdd-9334589ba71a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:00.422064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:00.424299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:00.425009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:00.425752 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:00.455904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:00.458999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:00.483311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1162/2280] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:24:00.514369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4804e980ecc84e12a90f01bca1acee0a', 'request_id': 'req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:00.523639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4804e980ecc84e12a90f01bca1acee0a', 'request_id': 'req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:00.531196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4804e980ecc84e12a90f01bca1acee0a', 'request_id': 'req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:00.551574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] The token cache is being invalidate because project 4804e980ecc84e12a90f01bca1acee0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:24:00.551872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:00.552664 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:00.554736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2281] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:24:00 2022] DELETE /identity/v3/projects/4804e980ecc84e12a90f01bca1acee0a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:24:00.989971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:00.992744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:01.018058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e0754-afa3-4764-a47e-b2bd8825eee8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:01.100447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:01.101127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:01.101412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:01.101695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] PATH_INFO: `/v3/projects/d8b159e657d843cb82aeddc3ac276128` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:01.102914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b159e657d843cb82aeddc3ac276128)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:01.106029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:01.165204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b159e657d843cb82aeddc3ac276128', 'request_id': 'req-45ff75d3-6fc7-4112-b182-d81f2bc1463e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.179171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b159e657d843cb82aeddc3ac276128', 'request_id': 'req-45ff75d3-6fc7-4112-b182-d81f2bc1463e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.188009 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b159e657d843cb82aeddc3ac276128', 'request_id': 'req-45ff75d3-6fc7-4112-b182-d81f2bc1463e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.200709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:01.207688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] The token cache is being invalidate because project d8b159e657d843cb82aeddc3ac276128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:24:01.212379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.214634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-45ff75d3-6fc7-4112-b182-d81f2bc1463e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.219449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1163/2282] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:24:00 2022] DELETE /identity/v3/projects/d8b159e657d843cb82aeddc3ac276128 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:24:01.223321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:01.225708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:01.241145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b69f35-fd2e-4134-ac89-52a0b08bb3fa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:01.323593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:01.325049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:01.325334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:01.325616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] PATH_INFO: `/v3/users/bd1e7d7f7a0f4b1ea89b4747d9824867` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:01.327005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1e7d7f7a0f4b1ea89b4747d9824867)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:01.328995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:01.402931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1e7d7f7a0f4b1ea89b4747d9824867'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.404849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1e7d7f7a0f4b1ea89b4747d9824867'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.415064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1e7d7f7a0f4b1ea89b4747d9824867'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.417728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1e7d7f7a0f4b1ea89b4747d9824867'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.432455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:01.449325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1e7d7f7a0f4b1ea89b4747d9824867'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:01.472602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2283] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:24:01 2022] DELETE /identity/v3/users/bd1e7d7f7a0f4b1ea89b4747d9824867 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:24:01.493505 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:01.493963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:01.494125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:01.519416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:01.521116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:01.607598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1164/2284] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:24:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:01.628151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:01.630281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:01.746131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2b68-11f7-48ce-a7ef-643f65ab87e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:01.755387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:01.756464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:01.757426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:01.757589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:01.785190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:01.786940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:01.800727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2285] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:24:01.867785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:01.872115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:02.180617 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c3056-b5c5-46d4-9883-78babd95a79a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:02.206128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:02.207498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:02.207789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:02.208070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] PATH_INFO: `/v3/projects/da46cb1cbf8b4a969fa0b13cd913b666` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:02.209401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da46cb1cbf8b4a969fa0b13cd913b666)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:02.212096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:02.311908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da46cb1cbf8b4a969fa0b13cd913b666', 'request_id': 'req-9b83b96c-2c50-405f-aed2-5fc264aee0f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.329589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da46cb1cbf8b4a969fa0b13cd913b666', 'request_id': 'req-9b83b96c-2c50-405f-aed2-5fc264aee0f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.350933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da46cb1cbf8b4a969fa0b13cd913b666', 'request_id': 'req-9b83b96c-2c50-405f-aed2-5fc264aee0f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.381832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] The token cache is being invalidate because project da46cb1cbf8b4a969fa0b13cd913b666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:24:02.382123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.388270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.390650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1165/2286] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:24:01 2022] DELETE /identity/v3/projects/da46cb1cbf8b4a969fa0b13cd913b666 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:24:02.395297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:02.414439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3f380-8215-4a73-a99c-996ddd939eaf service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:02.541570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:02.543125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:02.543747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:02.544327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] PATH_INFO: `/v3/users/419132e29a614919a0ba9b14e54a60c0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:02.546274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=419132e29a614919a0ba9b14e54a60c0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:02.548556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:02.649185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419132e29a614919a0ba9b14e54a60c0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.649874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419132e29a614919a0ba9b14e54a60c0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.670521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419132e29a614919a0ba9b14e54a60c0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.672263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419132e29a614919a0ba9b14e54a60c0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.683615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:02.708248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce8445e-53e3-491f-af33-4f32cf97bf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419132e29a614919a0ba9b14e54a60c0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:02.734014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2287] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:24:02 2022] DELETE /identity/v3/users/419132e29a614919a0ba9b14e54a60c0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:24:03.312672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:03.316238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:03.351409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b96c-2c50-405f-aed2-5fc264aee0f1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:03.414958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:03.467485 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:03.468509 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:03.468793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:03.469074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] PATH_INFO: `/v3/projects/4a2ce2657d6c41129fb16427f40ee56e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:03.470408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2ce2657d6c41129fb16427f40ee56e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:03.472894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:03.546390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2ce2657d6c41129fb16427f40ee56e', 'request_id': 'req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:03.574557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2ce2657d6c41129fb16427f40ee56e', 'request_id': 'req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:03.591555 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2ce2657d6c41129fb16427f40ee56e', 'request_id': 'req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:03.606219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] The token cache is being invalidate because project 4a2ce2657d6c41129fb16427f40ee56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:24:03.606684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:03.607483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3a841a6-43bc-4e8a-be1e-fe797cec4aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:24:03.609130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1166/2288] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:24:03 2022] DELETE /identity/v3/projects/4a2ce2657d6c41129fb16427f40ee56e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:24:03.617806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:03.618638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:03.618812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:03.652762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:03.654163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:03.767665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2289] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:03.776974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:03.777273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:03.777574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:03.815217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:03.817686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:03.831260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1167/2290] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:24:03.833102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:03.889142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551a661-1798-48f2-8c84-4d407c53b14e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:03.913833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:03.914688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:03.915080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:03.916916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:03.917862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:24:03.918303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:03.920175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:03.938424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2291] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:24:03.944176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:03.985449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42c2a0-7300-44b0-8d6b-b2f505d64ae4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:04.005751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:04.006415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:04.006775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:04.007304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:04.013687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:04.016974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:04.066444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1168/2292] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:24:03 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:24:04.067519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:04.125127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4bb98-b84b-4b7f-ae9e-c6dc66f13e29 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:04.376536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:04.378524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:04.379153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:04.379886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:04.381246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:04.385910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:04.461492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2293] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:24:04 2022] POST /identity/v3/users => generated 287 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:24:04.465256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:04.489946 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d77d08-00c6-4206-8b54-fc5e2e97e3e5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:04.499186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:04.500169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:04.500725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:04.501905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:04.503068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:04.506706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:04.521395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1169/2294] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:24:04.526938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:04.553012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea760a2-2ec3-405c-a44e-e28fd1e47cae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:04.568157 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858f840-6c0d-445a-af93-fcc71b9efea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:04.571502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858f840-6c0d-445a-af93-fcc71b9efea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:04.571502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858f840-6c0d-445a-af93-fcc71b9efea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:04.571770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858f840-6c0d-445a-af93-fcc71b9efea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:04.573876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858f840-6c0d-445a-af93-fcc71b9efea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:04.577459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858f840-6c0d-445a-af93-fcc71b9efea1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:04.605546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2295] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:24:04.610252 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:04.640125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241120-a3fc-40f2-bc6e-dac0c35e6bec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:04.654142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:04.654761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:04.655298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:04.655821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] PATH_INFO: `/v3/projects/b28baf3f14f74766b11c9ee459b8d5ec/users/27709d338705483b97997fcfd48d1d4a/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:04.720758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28baf3f14f74766b11c9ee459b8d5ec, user_id=27709d338705483b97997fcfd48d1d4a, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:04.730541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:04.780159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1170/2296] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:24:04 2022] PUT /identity/v3/projects/b28baf3f14f74766b11c9ee459b8d5ec/users/27709d338705483b97997fcfd48d1d4a/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:24:04.792237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b14e6-ba69-4796-87ee-f9ecca269705 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:04.793588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b14e6-ba69-4796-87ee-f9ecca269705 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:04.793932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b14e6-ba69-4796-87ee-f9ecca269705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:04.842197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e21b14e6-ba69-4796-87ee-f9ecca269705 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:04.843884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e21b14e6-ba69-4796-87ee-f9ecca269705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:04.916581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:04.966385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2297] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:04.986729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:04.992514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:05.118817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91382c-d752-45eb-9749-b615a21cff13 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:05.123958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:05.124780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:05.125195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:05.125478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:05.141023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:05.142557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:05.166751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1171/2298] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:24:06.563321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc60545-ad7d-4df8-9cd1-2d4b42c8b7b5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:06.564325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc60545-ad7d-4df8-9cd1-2d4b42c8b7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:06.564489 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc60545-ad7d-4df8-9cd1-2d4b42c8b7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:06.636759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3dc60545-ad7d-4df8-9cd1-2d4b42c8b7b5 None None] MFA Rules not processed for user `27709d338705483b97997fcfd48d1d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:06.638588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3dc60545-ad7d-4df8-9cd1-2d4b42c8b7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:06.784217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2299] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:24:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:06.798149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:06.798149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:06.803304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:06.946146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d75a1-2d50-4ccd-ae18-6eedd4d93a94 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:06.952679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:06.953502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:06.953773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:06.954053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:06.986227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:06.988322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:07.006917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1172/2300] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:24:08.669568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208a1fd-482c-4024-9e14-336c950c6345 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:08.670122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208a1fd-482c-4024-9e14-336c950c6345 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:08.670420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208a1fd-482c-4024-9e14-336c950c6345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:08.718635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4208a1fd-482c-4024-9e14-336c950c6345 None None] MFA Rules not processed for user `27709d338705483b97997fcfd48d1d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:08.720873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4208a1fd-482c-4024-9e14-336c950c6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:08.741873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2301] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:24:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:09.543708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:09.545342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:09.658090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94baea00-8e6a-4a75-ad69-327f15e39831 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:09.664743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:09.666238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:09.668339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:09.668780 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:09.671089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:09.698580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:09.701643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c4b63-57bf-4151-9f18-aa3f7e414cda service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:09.723404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1173/2302] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:24:18.696040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=42 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:19.713803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=56 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:20.895951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71395e-f5d8-4906-af13-062f6d7d13bf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:20.896859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71395e-f5d8-4906-af13-062f6d7d13bf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:20.897073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71395e-f5d8-4906-af13-062f6d7d13bf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:20.900568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2303] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:24:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:24:20.907481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b546-4750-4245-bf23-bca34faaca64 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:20.908581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b546-4750-4245-bf23-bca34faaca64 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:20.908581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b546-4750-4245-bf23-bca34faaca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:20.934533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d2d7b546-4750-4245-bf23-bca34faaca64 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:20.935879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d2d7b546-4750-4245-bf23-bca34faaca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:21.029978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1174/2304] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:24:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:24:21.043970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71395e-f5d8-4906-af13-062f6d7d13bf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:21.045973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c71395e-f5d8-4906-af13-062f6d7d13bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:21.111100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71395e-f5d8-4906-af13-062f6d7d13bf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:21.114690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcedca-d190-4004-bd35-73ee841cb4a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:21.115318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcedca-d190-4004-bd35-73ee841cb4a7 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.115634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcedca-d190-4004-bd35-73ee841cb4a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.115914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcedca-d190-4004-bd35-73ee841cb4a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.127959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcedca-d190-4004-bd35-73ee841cb4a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:21.129412 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcedca-d190-4004-bd35-73ee841cb4a7 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:21.143218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2305] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:24:21.222909 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a436dac-d7a7-4984-9764-977de8db48ee None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.223280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a436dac-d7a7-4984-9764-977de8db48ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.223518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a436dac-d7a7-4984-9764-977de8db48ee None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.224893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1175/2306] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:24:21.230671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12b7cc-5832-4d68-8cd1-b60cc1e06ef3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.232162 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12b7cc-5832-4d68-8cd1-b60cc1e06ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.232501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12b7cc-5832-4d68-8cd1-b60cc1e06ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.255197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f12b7cc-5832-4d68-8cd1-b60cc1e06ef3 None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:21.256880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f12b7cc-5832-4d68-8cd1-b60cc1e06ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:21.266728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2307] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:24:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:21.281301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a436dac-d7a7-4984-9764-977de8db48ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:21.294816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a436dac-d7a7-4984-9764-977de8db48ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:21.298772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839e94-e65a-4017-8024-47b6095e563e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:21.299450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839e94-e65a-4017-8024-47b6095e563e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.299775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839e94-e65a-4017-8024-47b6095e563e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.300096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839e94-e65a-4017-8024-47b6095e563e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.309348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58839e94-e65a-4017-8024-47b6095e563e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:21.311039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58839e94-e65a-4017-8024-47b6095e563e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:21.320538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1176/2308] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:24:21.414564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4863db-580e-4c70-be77-6c1d1c4e00f0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.415785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4863db-580e-4c70-be77-6c1d1c4e00f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.416602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4863db-580e-4c70-be77-6c1d1c4e00f0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.417553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2309] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:24:21.424047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20394bd6-7c5b-446d-a111-0eb484778b59 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.424481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20394bd6-7c5b-446d-a111-0eb484778b59 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.424979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20394bd6-7c5b-446d-a111-0eb484778b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.448676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-20394bd6-7c5b-446d-a111-0eb484778b59 None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:21.449981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20394bd6-7c5b-446d-a111-0eb484778b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:21.460871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1177/2310] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:24:21.473434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4863db-580e-4c70-be77-6c1d1c4e00f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:21.475252 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df4863db-580e-4c70-be77-6c1d1c4e00f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:21.491710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4863db-580e-4c70-be77-6c1d1c4e00f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:21.496114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec0504-8742-4e82-9297-4694aab24aeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:21.497138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0504-8742-4e82-9297-4694aab24aeb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:21.497138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0504-8742-4e82-9297-4694aab24aeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:21.497488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0504-8742-4e82-9297-4694aab24aeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:21.509812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0504-8742-4e82-9297-4694aab24aeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:21.512499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0504-8742-4e82-9297-4694aab24aeb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:21.533499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2311] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:24:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:24:31.442167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=40 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:31.523526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=35 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:33.668163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:33.668715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:33.669161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:33.701978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] MFA Rules not processed for user `27709d338705483b97997fcfd48d1d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:33.703279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:33.711594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1178/2312] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:24:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:24:34.992889 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d19a2-6603-455c-923a-5cb50287077b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:34.993988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d19a2-6603-455c-923a-5cb50287077b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:34.993988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d19a2-6603-455c-923a-5cb50287077b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:35.018067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e56d19a2-6603-455c-923a-5cb50287077b None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:24:35.019520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e56d19a2-6603-455c-923a-5cb50287077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:35.116354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2313] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:24:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:24:35.131287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:24:35.133651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:24:35.249835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da01ad5-7a73-4804-a5ea-805c6faa767f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:24:35.255464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bacdd7-afe0-40ff-9dca-cc303611fd9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:24:35.256119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bacdd7-afe0-40ff-9dca-cc303611fd9e service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:24:35.256406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bacdd7-afe0-40ff-9dca-cc303611fd9e service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:24:35.256685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bacdd7-afe0-40ff-9dca-cc303611fd9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:24:35.268543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bacdd7-afe0-40ff-9dca-cc303611fd9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:24:35.270220 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bacdd7-afe0-40ff-9dca-cc303611fd9e service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:24:35.282073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1179/2314] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 05:24:45.103697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:24:45.276552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=35 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:25:04.201396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:04.202475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:04.202975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:04.228566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] MFA Rules not processed for user `1690be2d7032472391fa96e23e0c1bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:25:04.230049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:25:04.361733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2315] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:25:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:25:05.860618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95c2d-bb63-4e1a-bb47-bccba34506b4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:05.861640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95c2d-bb63-4e1a-bb47-bccba34506b4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:05.862170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95c2d-bb63-4e1a-bb47-bccba34506b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:05.901644 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-46c95c2d-bb63-4e1a-bb47-bccba34506b4 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:25:05.903008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46c95c2d-bb63-4e1a-bb47-bccba34506b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:25:05.912871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1180/2316] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 05:25:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:25:05.942180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:25:05.961257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8471-230c-4b2a-910b-5c7b7c751775 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:25:05.969702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b7833-973a-4f58-b0af-64dd23522cbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:25:05.970792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b7833-973a-4f58-b0af-64dd23522cbc service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:05.971249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b7833-973a-4f58-b0af-64dd23522cbc service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:05.971760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b7833-973a-4f58-b0af-64dd23522cbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:05.981948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b7833-973a-4f58-b0af-64dd23522cbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:25:05.983890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b7833-973a-4f58-b0af-64dd23522cbc service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:25:06.002199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2317] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:25:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:25:14.733520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:14.734440 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:14.734607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:14.756041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] MFA Rules not processed for user `3c60b3a983f14f50b1b762a3954a1059`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:25:14.757301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:25:14.820257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1181/2318] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:25:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:25:15.990484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=31 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:25:18.539750 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca58a-01f1-4123-abd8-c1bbdc0a782b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:18.540426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca58a-01f1-4123-abd8-c1bbdc0a782b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:18.540426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca58a-01f1-4123-abd8-c1bbdc0a782b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:18.561207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0efca58a-01f1-4123-abd8-c1bbdc0a782b None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:25:18.562539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0efca58a-01f1-4123-abd8-c1bbdc0a782b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:25:18.573784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2319] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:25:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:25:18.585257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:25:18.597916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3976800-ab7e-4621-a74c-52d617ff9b00 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:25:18.609910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c47b1-741d-4953-ab2a-7f1d6a9f0330 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:25:18.610666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c47b1-741d-4953-ab2a-7f1d6a9f0330 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:18.610959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c47b1-741d-4953-ab2a-7f1d6a9f0330 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:18.611294 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c47b1-741d-4953-ab2a-7f1d6a9f0330 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:18.621844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c47b1-741d-4953-ab2a-7f1d6a9f0330 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:25:18.623510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c47b1-741d-4953-ab2a-7f1d6a9f0330 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:25:18.634591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1182/2320] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:25:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 05:25:28.557049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:25:28.635232 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=34 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:25:31.089785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:31.090325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:31.090408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:31.115024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] MFA Rules not processed for user `45ae88234e984149be8539757efa8733`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:25:31.116306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:25:31.203580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2321] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:25:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:25:38.319524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:38.320010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:38.320097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:38.339228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:25:38.340480 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:25:38.348386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1183/2322] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 05:25:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:25:38.362346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:25:38.377837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736bc5-e027-49b2-9165-788232e77672 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:25:38.383958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:25:38.384612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:38.384898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:38.385176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:38.396044 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:25:38.397691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:25:38.407303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2323] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:25:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:25:43.899019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:25:43.915888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c72b-7da2-44d4-b3f1-85ee99c3cd6a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:25:43.921889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:25:43.923330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:43.923734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:43.924095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] PATH_INFO: `/v3/users/27709d338705483b97997fcfd48d1d4a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:43.926428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=27709d338705483b97997fcfd48d1d4a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:25:43.929468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:25:44.016266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27709d338705483b97997fcfd48d1d4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.025325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27709d338705483b97997fcfd48d1d4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.026240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27709d338705483b97997fcfd48d1d4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.026976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27709d338705483b97997fcfd48d1d4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.043954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27709d338705483b97997fcfd48d1d4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.063396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1184/2324] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:25:43 2022] DELETE /identity/v3/users/27709d338705483b97997fcfd48d1d4a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:25:44.272336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:25:44.299208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7b9d-97d0-4f87-9d1f-549b3e59634d service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:25:44.373385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:25:44.374309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:25:44.374665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:25:44.375009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] PATH_INFO: `/v3/projects/b28baf3f14f74766b11c9ee459b8d5ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:25:44.376502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28baf3f14f74766b11c9ee459b8d5ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:25:44.379132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:25:44.444300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28baf3f14f74766b11c9ee459b8d5ec', 'request_id': 'req-eb288dd9-a8e8-4323-aca4-318dfb49d13f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.452283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28baf3f14f74766b11c9ee459b8d5ec', 'request_id': 'req-eb288dd9-a8e8-4323-aca4-318dfb49d13f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.460400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28baf3f14f74766b11c9ee459b8d5ec', 'request_id': 'req-eb288dd9-a8e8-4323-aca4-318dfb49d13f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.479153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] The token cache is being invalidate because project b28baf3f14f74766b11c9ee459b8d5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:25:44.479520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.480704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eb288dd9-a8e8-4323-aca4-318dfb49d13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:25:44.482446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2325] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:25:44 2022] DELETE /identity/v3/projects/b28baf3f14f74766b11c9ee459b8d5ec => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:25:54.055205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:25:54.483251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:02.012125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:02.013598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:02.073054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f925a-0d0f-4371-b89e-580643b614bb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:02.123383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:02.124003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:02.124291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:02.124572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:02.126346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:02.201181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:02.202637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:02.215171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1185/2326] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:09.011817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:09.012321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:09.012434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:09.035312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] MFA Rules not processed for user `af5a109a6fb94042aeacbf141d7322ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:09.036691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:09.099138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2327] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:09.792796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:09.794347 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:09.807570 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7baa5b-e676-4fff-ae37-4dec3616bebf service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:09.816649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:09.817295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:09.817579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:09.817864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:09.819690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:09.835018 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:09.836605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:09.849911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1186/2328] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:15.867141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:15.868809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:15.963283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9469df2-809a-4df7-8ebd-409d73c4294f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:15.970069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:15.971472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:15.971965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:15.972388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] PATH_INFO: `/v3/users/1690be2d7032472391fa96e23e0c1bf0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:15.974274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1690be2d7032472391fa96e23e0c1bf0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:15.976561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:16.046022 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1690be2d7032472391fa96e23e0c1bf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.047089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1690be2d7032472391fa96e23e0c1bf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.056451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1690be2d7032472391fa96e23e0c1bf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.057240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1690be2d7032472391fa96e23e0c1bf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.072216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a68d79e3-8c11-4322-a607-050865d337ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1690be2d7032472391fa96e23e0c1bf0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.090573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2329] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:26:15 2022] DELETE /identity/v3/users/1690be2d7032472391fa96e23e0c1bf0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:26:16.373469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:16.397115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3480bb4-4f40-4ba4-8ae7-8d99e67489d2 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:16.475218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:16.496173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:16.497388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:16.497906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:16.498445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] PATH_INFO: `/v3/projects/1f75331a524c4a5a981dfd1fdc361467` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:16.500856 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f75331a524c4a5a981dfd1fdc361467)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:16.504501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:16.571842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f75331a524c4a5a981dfd1fdc361467', 'request_id': 'req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.582155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f75331a524c4a5a981dfd1fdc361467', 'request_id': 'req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.590065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f75331a524c4a5a981dfd1fdc361467', 'request_id': 'req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.603321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] The token cache is being invalidate because project 1f75331a524c4a5a981dfd1fdc361467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:26:16.603643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.604312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f49fdc4e-b20a-4cb2-8d66-dade6bee701f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:26:16.605850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1187/2330] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:26:16 2022] DELETE /identity/v3/projects/1f75331a524c4a5a981dfd1fdc361467 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:26:16.611891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:16.612268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:16.612562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:16.632781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:16.634025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:16.655610 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:16.700937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2331] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:16.711452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:16.711954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:16.712519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:16.747231 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:16.749415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:16.763452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1188/2332] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:26:16.768705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:16.786807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59533bb4-f8af-44b1-bda4-b56fee18cf85 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:16.794913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:16.796044 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:16.796553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:16.797060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:16.798370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:26:16.799716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:16.802279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:16.829548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2333] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:26:16.831587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:16.843012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5a703-4e0e-47cb-bd08-3c62ea8b4b77 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:16.858173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:16.860283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:16.860794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:16.861317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:16.862601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:16.870929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:16.923293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1189/2334] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:26:16 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:26:16.925881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:16.936883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad04d6-2455-4a5f-ab73-7bd85f5bad47 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:17.013977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:17.014948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:17.015440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:17.015856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:17.016852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:17.020167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:17.053288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2335] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:26:16 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:26:17.059887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:17.079332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f6b0-0fe9-4cf0-a939-717220f6a776 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:17.094638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:17.096462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:17.096831 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:17.097545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:17.098679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:17.100843 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:17.115618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1190/2336] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:26:17.116932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:17.129402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7626e-b8a3-475b-94c7-e495c3fd37e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:17.134468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187e47-43ae-4a22-90dd-7c8223285ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:17.135102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187e47-43ae-4a22-90dd-7c8223285ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:17.135419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187e47-43ae-4a22-90dd-7c8223285ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:17.135708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187e47-43ae-4a22-90dd-7c8223285ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:17.136419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187e47-43ae-4a22-90dd-7c8223285ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:17.138060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187e47-43ae-4a22-90dd-7c8223285ab3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:17.150559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2337] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:26:17.155281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:17.167750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6332ce-2ff2-4be3-9ae2-10e057e4e283 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:17.180049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:17.180748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:17.181033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:17.181308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] PATH_INFO: `/v3/projects/4c28b78169ac49a6854f195bea0dd9bf/users/190f6130be544a73bb619f7b49ab4e87/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:17.201877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c28b78169ac49a6854f195bea0dd9bf, user_id=190f6130be544a73bb619f7b49ab4e87, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:17.204718 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:17.223892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1191/2338] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:26:17 2022] PUT /identity/v3/projects/4c28b78169ac49a6854f195bea0dd9bf/users/190f6130be544a73bb619f7b49ab4e87/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:26:17.228964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:17.229366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:17.229727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:17.251332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:17.253085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:17.352039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2339] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:17.393004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:17.395333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:17.462524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1758d63-c52e-408d-9c2e-6ef3e933c755 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:17.466216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:17.466895 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:17.467181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:17.467488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:17.478326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:17.480326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:17.490992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1192/2340] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:21.652603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:21.654849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:21.766206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:21.823375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b8d10-157c-4aec-a790-47fa5006b20f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:21.830302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:21.831432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:21.831899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:21.832340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:21.834550 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:21.938702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:21.941187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95081d4f-72ef-4a1c-965b-f8554fd576e4 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:21.971487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2341] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:24.556960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:24.558611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:24.603716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:24.701944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404faf7-4c92-49a9-a37b-8f84c335feed service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:24.715754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:24.717265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:24.717559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:24.717859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:24.723929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:24.875458 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:24.879910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23140-c00a-4601-bc66-360d3a91ce25 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:24.897053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1193/2342] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:25.264490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:25.265098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:25.265098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:25.303154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:25.303774 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:25.304282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:25.371668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] MFA Rules not processed for user `c2db6ada97da4ee89f3ce8de98f2653a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:25.373033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:25.396054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] MFA Rules not processed for user `190f6130be544a73bb619f7b49ab4e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:25.400168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:25.511262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1194/2343] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:26:25.661808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2344] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:25.666155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:25.683667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae483c-21c2-4ba8-8d2f-23835b85de0f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:25.693583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:25.694512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:25.695074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:25.695605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:25.696545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:25.699114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:25.741309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1195/2345] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:26:25 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:26:25.746264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:25.767490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c993b4-fdea-4f8a-a3d9-76e732a5ed33 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:25.899062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:25.910316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:25.911261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:25.911635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:25.911901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:25.912568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:25.914638 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:25.946183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2346] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:26:25 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:26:25.951006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:25.972755 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024eb505-0570-4451-b411-7513ef82c7ff admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:25.990129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:25.991087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:25.993181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:25.993964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:25.995947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:25.998387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:26.013427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1196/2347] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:26:26.020814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:26.039970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e56e5-6c72-487a-be4c-15a7ede3db5b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:26.053646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:26.055575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:26.056054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:26.056788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] PATH_INFO: `/v3/projects/5f45c39eb99549d58fd35a248f6cbac6/users/1febec84e89b43cd8af72a7ed01e5e68/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:26.071730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:26.074019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:26.091537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f45c39eb99549d58fd35a248f6cbac6, user_id=1febec84e89b43cd8af72a7ed01e5e68, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:26.098289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:26.124340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2348] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:26:26 2022] PUT /identity/v3/projects/5f45c39eb99549d58fd35a248f6cbac6/users/1febec84e89b43cd8af72a7ed01e5e68/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:26:26.130689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:26.151398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:26.178272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8a10-f06d-4f59-9a59-e7da0079cf30 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:26.233363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21036ea2-e719-48af-b5b2-bdb8035734d4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:26.242106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:26.243422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:26.243732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:26.244590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:26.245354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:26.247061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:26.258616 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:26.263096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2349] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:26:26.263702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:26.264017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:26.264169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:26.264304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:26.266504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:26.277556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0cac7-3d16-4d0c-a33f-5b5f0ac8f08c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:26.278642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:26.280482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:26.286650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:26.287980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:26.288492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:26.289186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] PATH_INFO: `/v3/projects/5f45c39eb99549d58fd35a248f6cbac6/users/1febec84e89b43cd8af72a7ed01e5e68/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:26.290184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1197/2350] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:26.291799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f45c39eb99549d58fd35a248f6cbac6, user_id=1febec84e89b43cd8af72a7ed01e5e68, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:26.294811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:26.308567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2351] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:26:26 2022] PUT /identity/v3/projects/5f45c39eb99549d58fd35a248f6cbac6/users/1febec84e89b43cd8af72a7ed01e5e68/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:26:26.908030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:26.913285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:27.118637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e071d91-f91d-4cd5-a2ba-f9c0c56420f5 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:27.237475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:27.239052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:27.239604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:27.240156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:27.242867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:27.260104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:27.262280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbe8a5-6425-4cef-a2f2-ef2ade46bf78 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:27.281666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1198/2352] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:30.385981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:30.504996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac0c6-3ff5-4e6c-b6a6-46c5620ffb2a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:30.519729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:30.520644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:30.521910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:30.522450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:30.524642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:30.604680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:30.636245 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:30.639111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:30.661651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2353] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:32.880839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd20d5-e1ab-4963-b7f7-bf3c7d0d98b2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:32.881407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd20d5-e1ab-4963-b7f7-bf3c7d0d98b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:32.881407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd20d5-e1ab-4963-b7f7-bf3c7d0d98b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:32.921891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f0dd20d5-e1ab-4963-b7f7-bf3c7d0d98b2 None None] MFA Rules not processed for user `1febec84e89b43cd8af72a7ed01e5e68`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:32.923276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0dd20d5-e1ab-4963-b7f7-bf3c7d0d98b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:32.954123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:33.021137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1199/2354] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:33.039242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:33.040759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:33.116820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826293eb-8918-42e7-9eb5-1ad74d09056c service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:33.120823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:33.121561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:33.121914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:33.122263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:33.192014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:33.193745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:33.204574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2355] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:26:33.334435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:33.339948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:33.340266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:33.387383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:33.388645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:33.488956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1200/2356] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:26:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:33.523454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:33.526354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:33.599528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27743f-3d2b-46e8-beaf-0eaf74f19cd4 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:33.604773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:33.605426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:33.605741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:33.606021 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:33.616148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:33.617853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:33.629419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2357] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:26:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:26:36.234894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:36.372809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0970880-e366-4355-b106-41ce149eafd4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:36.380980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:36.381638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:36.381926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:36.382319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:36.395042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:36.427334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:36.431729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1cb7d-5438-44b5-988a-b1271a68d1a0 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:36.471557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1201/2358] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:43.004934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:43.083818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6e52-cdcd-41a0-82d2-ecc31e0c0701 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:43.104067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:43.105105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:43.105587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:43.106072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:43.112165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:43.165039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:43.167139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab86f3-aeef-4495-bdaa-a989c96a0a00 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:43.197141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2359] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:26:46.445678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=71 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:47.061101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:47.061584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:47.061987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:47.088250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] MFA Rules not processed for user `3c60b3a983f14f50b1b762a3954a1059`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:47.089781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:47.210447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1202/2360] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:47.890269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cf9f-5f36-4e08-b189-2190e9119b39 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:47.891599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cf9f-5f36-4e08-b189-2190e9119b39 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:47.891599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cf9f-5f36-4e08-b189-2190e9119b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:47.912930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ab01cf9f-5f36-4e08-b189-2190e9119b39 None None] MFA Rules not processed for user `190f6130be544a73bb619f7b49ab4e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:47.914225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab01cf9f-5f36-4e08-b189-2190e9119b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:47.922478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2361] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:26:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:48.245982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:48.257819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6576399-e0d6-4e3c-a5a6-d448c50f38dd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:48.261999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:48.262645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:48.262925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:48.263201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:48.265588 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:48.281529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:48.282861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:48.294396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1203/2362] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:26:50.507589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:50.508530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:50.508832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:50.526088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:26:50.536142 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:26:50.537694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:50.546422 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2363] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:26:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:26:50.564863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:50.581026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f581-9eef-4d35-aabc-5ea30dacae3b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:50.589265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:50.590379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:50.590922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:50.591465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:50.608406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:50.612450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:50.629941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1204/2364] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 05:26:59.614800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:26:59.617501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:59.652913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd7d1e-42f7-49e5-a8af-1fe4232b8bcb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:26:59.662902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:26:59.664006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:26:59.664504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:26:59.664985 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:26:59.668588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:26:59.767700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:26:59.770839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:26:59.782464 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2365] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:26:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:00.634590 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=37 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:06.808701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:06.818434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eac53b-b414-4500-9e58-de5c5fa1ac14 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:06.822697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:06.823339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:06.823685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:06.823945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:06.826635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:06.931572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:06.933076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:06.945135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1205/2366] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:27:08.887523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:08.914768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48212e-c7b9-4ee5-b8d1-a6dff93b4b3d service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:08.926520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:08.927800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:08.929080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:08.929697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:08.933259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:08.967174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:08.974478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:08.990644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2367] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:12.051771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:12.053370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:12.078377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb448f-ea4b-4a6b-b6d5-c2566ae72132 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:12.094344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:12.097258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:12.098049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:12.099051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] PATH_INFO: `/v3/users/45ae88234e984149be8539757efa8733` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:12.102746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ae88234e984149be8539757efa8733)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:12.106121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:12.211577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ae88234e984149be8539757efa8733'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.230362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ae88234e984149be8539757efa8733'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.231663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ae88234e984149be8539757efa8733'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.232787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ae88234e984149be8539757efa8733'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.259016 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c11553d-a9b3-4f2a-a1b7-f8d09c8546d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ae88234e984149be8539757efa8733'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.286195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1206/2368] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:12 2022] DELETE /identity/v3/users/45ae88234e984149be8539757efa8733 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:12.540482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:12.565081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c822f9-ded8-494f-badf-ff3e47e079d5 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:12.655412 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:12.656107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:12.656396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:12.656678 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] PATH_INFO: `/v3/projects/153fe5d9d64346f6b7419eba91948678` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:12.658902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] RBAC: Authorizing `identity:delete_project(project_id=153fe5d9d64346f6b7419eba91948678)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:12.661346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:12.719588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153fe5d9d64346f6b7419eba91948678', 'request_id': 'req-7b0714e8-3717-4b7b-bc4f-44f62d69708f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.727744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153fe5d9d64346f6b7419eba91948678', 'request_id': 'req-7b0714e8-3717-4b7b-bc4f-44f62d69708f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.736858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153fe5d9d64346f6b7419eba91948678', 'request_id': 'req-7b0714e8-3717-4b7b-bc4f-44f62d69708f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.752999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] The token cache is being invalidate because project 153fe5d9d64346f6b7419eba91948678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:12.752999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.753818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7b0714e8-3717-4b7b-bc4f-44f62d69708f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:12.755951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2369] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:12 2022] DELETE /identity/v3/projects/153fe5d9d64346f6b7419eba91948678 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:12.762557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:12.763292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:12.763864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:12.797714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:12.799607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:12.896246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1207/2370] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:12.905819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:12.906354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:12.906846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:12.942374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:12.944703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:12.959557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2371] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:12.966172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:12.998781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c287-a946-4666-90f0-011b0019d2d1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.008433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.009518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.010086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.010706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.012272 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:27:13.012959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.016403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.020282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:13.039413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1208/2372] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:27:13.042623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:13.053769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a3331-2e5b-4cab-9672-0e42455b9cce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.071609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.072358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.072713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.073123 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.073937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.076119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.084745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:13.135728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2373] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:13 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:13.138650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:13.172270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4c8fa-b02f-45c4-9b92-53408d648ea0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.343914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.346313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.347227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.348649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.350390 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.354377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.396285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1209/2374] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:13 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:13.401390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:13.414683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d1c76-8d72-44d7-be0e-9a95f53395e8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.424263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.425693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.426298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.426810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.428191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.431387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.447473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2375] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:13.453332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:13.466218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec249a-5d1c-417d-bd11-ba3834c105a9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.473146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b9212-c564-477b-85ba-b7108017a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.473908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b9212-c564-477b-85ba-b7108017a6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.474270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b9212-c564-477b-85ba-b7108017a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.474623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b9212-c564-477b-85ba-b7108017a6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.475524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b9212-c564-477b-85ba-b7108017a6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.477314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b9212-c564-477b-85ba-b7108017a6b0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.499198 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1210/2376] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:13.504946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:13.523549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc5c96-b88e-4bd5-b3db-643684f18d98 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.531961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.533804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.534300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.534809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] PATH_INFO: `/v3/projects/b80e9efe673244a79cf0270be1323e0e/users/7da79af963a84c07aae68d86aadbcd4f/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.562188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80e9efe673244a79cf0270be1323e0e, user_id=7da79af963a84c07aae68d86aadbcd4f, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.567126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.587972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2377] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:13 2022] PUT /identity/v3/projects/b80e9efe673244a79cf0270be1323e0e/users/7da79af963a84c07aae68d86aadbcd4f/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:13.609146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.609436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.609899 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.661977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] MFA Rules not processed for user `7da79af963a84c07aae68d86aadbcd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:13.663605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:13.778560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1211/2378] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:13.796008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:13.798204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:13.903583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b521d-7048-4d86-9398-3f1effc3d1d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:13.916523 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:13.917660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:13.918173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:13.918668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:13.935859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:13.939543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:13.957492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:15.541259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:15.543929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:15.568995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a643801-f8ce-454a-9d92-012090d83ee7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:15.577491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:15.578455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:15.578992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:15.579444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:15.582271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:15.598551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:15.600273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7b87-a3b1-4011-aabe-439c7308dfb5 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:15.613742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1212/2380] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:22.042261 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:22.111924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d36bc4-cf20-43e8-b211-b6d5bc6ec829 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:22.117009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:22.117611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:22.117902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:22.118187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] PATH_INFO: `/v3/users/7da79af963a84c07aae68d86aadbcd4f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:22.119482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da79af963a84c07aae68d86aadbcd4f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:22.121463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:22.177854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:22.188933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da79af963a84c07aae68d86aadbcd4f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:22.189873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da79af963a84c07aae68d86aadbcd4f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:22.198907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da79af963a84c07aae68d86aadbcd4f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:22.200001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da79af963a84c07aae68d86aadbcd4f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:22.217376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da79af963a84c07aae68d86aadbcd4f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:22.233344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2381] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:22 2022] DELETE /identity/v3/users/7da79af963a84c07aae68d86aadbcd4f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:22.239538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:22.240215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:22.240215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:22.317901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:22.319240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:22.331238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:22.448172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1213/2382] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:22.460888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:22.463135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:22.545141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcb4cc-ff5f-4bba-8d20-ac6fc8a28660 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:22.552201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f1fad-8377-465d-8744-af9323c6cebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:22.553043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f1fad-8377-465d-8744-af9323c6cebf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:22.553396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f1fad-8377-465d-8744-af9323c6cebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:22.553886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f1fad-8377-465d-8744-af9323c6cebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:22.569626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f1fad-8377-465d-8744-af9323c6cebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:22.571113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f1fad-8377-465d-8744-af9323c6cebf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:22.583222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2383] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:23.165222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.166813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:23.192573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc499080-b126-4707-adc4-cc3b4041d21b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.212058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.213345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.213951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.214741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] PATH_INFO: `/v3/projects/b80e9efe673244a79cf0270be1323e0e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.217311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80e9efe673244a79cf0270be1323e0e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.221444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:23.300341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80e9efe673244a79cf0270be1323e0e', 'request_id': 'req-961f4d28-66b0-4b77-9f00-c13da505c0b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:23.316102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80e9efe673244a79cf0270be1323e0e', 'request_id': 'req-961f4d28-66b0-4b77-9f00-c13da505c0b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:23.327630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80e9efe673244a79cf0270be1323e0e', 'request_id': 'req-961f4d28-66b0-4b77-9f00-c13da505c0b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:23.342827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] The token cache is being invalidate because project b80e9efe673244a79cf0270be1323e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:23.343192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:23.343881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:23.346201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1214/2384] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:23 2022] DELETE /identity/v3/projects/b80e9efe673244a79cf0270be1323e0e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:23.351651 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb348b-25fd-41aa-8b8a-08cf698b875d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.352233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb348b-25fd-41aa-8b8a-08cf698b875d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.352233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb348b-25fd-41aa-8b8a-08cf698b875d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.423614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-81bb348b-25fd-41aa-8b8a-08cf698b875d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:23.425353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81bb348b-25fd-41aa-8b8a-08cf698b875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:23.542358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2385] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:23.550126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.551704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.552218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.552508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.553296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:23.584976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:23.586500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:23.605538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2386] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:23.611376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.629467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5e0f8-f94e-419f-8def-77bcfad3bfbf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.637662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.638326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.639767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.640244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.640986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:27:23.641347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.643613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:23.656704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:23.657030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2387] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:27:23.661774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.679321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21745083-c4f4-4d3a-8ce3-08ef874bde19 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.686989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.687675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.687963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.688243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.689478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.691931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:23.705660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:23.710873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4d28-66b0-4b77-9f00-c13da505c0b7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.717932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.719301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.719878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.720384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.723948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:23.759415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2388] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:23 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:23.766092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.842332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.845121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:23.858233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64b12-5bcc-4324-a625-7e2be90dd443 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.863117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1215/2389] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:23.864955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.865996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.866420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.866882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.867756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.869875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:23.899486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2390] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:23 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:27:23.903888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.924028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e76d-f299-40c1-ae69-c1a2c7686f13 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.932529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.933704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.934681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.935205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.937425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.940690 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:23.963515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1216/2391] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:23.969271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:23.981350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3fb6-6701-4f4c-be50-ae9e933258ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:23.987546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4e5cd-934e-402d-9d7e-fceeeb4b629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:23.988199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e5cd-934e-402d-9d7e-fceeeb4b629f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:23.988519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e5cd-934e-402d-9d7e-fceeeb4b629f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:23.988813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e5cd-934e-402d-9d7e-fceeeb4b629f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:23.989632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4e5cd-934e-402d-9d7e-fceeeb4b629f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:23.991399 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4e5cd-934e-402d-9d7e-fceeeb4b629f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:24.003161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2392] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:24.008091 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:24.020553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51f68-5e56-4105-8160-147b59639d77 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:24.026059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:24.027268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:24.027938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:24.028508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] PATH_INFO: `/v3/projects/4cd4d5a63dba4a98a8d496e70f3c34fe/users/fbd5075bcaa445e9973e82609090f4e1/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:24.047264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd4d5a63dba4a98a8d496e70f3c34fe, user_id=fbd5075bcaa445e9973e82609090f4e1, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:24.050416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:24.066474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1217/2393] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:24 2022] PUT /identity/v3/projects/4cd4d5a63dba4a98a8d496e70f3c34fe/users/fbd5075bcaa445e9973e82609090f4e1/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:24.077518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:24.077838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:24.078166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:24.135165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] MFA Rules not processed for user `fbd5075bcaa445e9973e82609090f4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:24.137671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:24.235027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2394] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:24.247370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:24.249299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:24.341542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3e6dc-7d82-4954-b0ef-19b65e4931da admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:24.348392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2311b14-1274-4ac1-9e0f-0ef61973094c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:24.349049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2311b14-1274-4ac1-9e0f-0ef61973094c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:24.349343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2311b14-1274-4ac1-9e0f-0ef61973094c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:24.349629 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2311b14-1274-4ac1-9e0f-0ef61973094c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:24.358474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2311b14-1274-4ac1-9e0f-0ef61973094c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:24.360242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2311b14-1274-4ac1-9e0f-0ef61973094c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:24.371473 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1218/2395] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:24.613792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:24.688447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:24.708546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314466-2c8d-4a31-899d-bda95725b0ac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:24.714659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:24.715476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:24.715760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:24.716145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] PATH_INFO: `/v3/users/fbd5075bcaa445e9973e82609090f4e1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:24.717781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd5075bcaa445e9973e82609090f4e1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:24.720982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:24.796326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd5075bcaa445e9973e82609090f4e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:24.797130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd5075bcaa445e9973e82609090f4e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:24.810820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd5075bcaa445e9973e82609090f4e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:24.811618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd5075bcaa445e9973e82609090f4e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:24.841128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd5075bcaa445e9973e82609090f4e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:24.865421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2396] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:24 2022] DELETE /identity/v3/users/fbd5075bcaa445e9973e82609090f4e1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:24.869721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:24.870013 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:24.870466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:24.902995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:24.904316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:24.949968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:25.036973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1219/2397] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:25.062439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:25.065001 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:25.171668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff1b3-0724-4243-ac1e-db0a243082f4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:25.184575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e36ef-09a1-40cc-8d39-31e50a36210f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:25.185241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e36ef-09a1-40cc-8d39-31e50a36210f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:25.185524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e36ef-09a1-40cc-8d39-31e50a36210f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:25.186158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e36ef-09a1-40cc-8d39-31e50a36210f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:25.204686 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e36ef-09a1-40cc-8d39-31e50a36210f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:25.206382 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e36ef-09a1-40cc-8d39-31e50a36210f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:25.218741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2398] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:25.626250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:25.629240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:25.653416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efc4c-bb58-4b1c-ab8e-b5a52cb8b4c6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:25.663051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:25.664241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:25.664875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:25.665331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] PATH_INFO: `/v3/projects/4cd4d5a63dba4a98a8d496e70f3c34fe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:25.667964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd4d5a63dba4a98a8d496e70f3c34fe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:25.672210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:25.744698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd4d5a63dba4a98a8d496e70f3c34fe', 'request_id': 'req-fc5324e7-2643-4903-a923-0d694f5a3744'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:25.754523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd4d5a63dba4a98a8d496e70f3c34fe', 'request_id': 'req-fc5324e7-2643-4903-a923-0d694f5a3744'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:25.762972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd4d5a63dba4a98a8d496e70f3c34fe', 'request_id': 'req-fc5324e7-2643-4903-a923-0d694f5a3744'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:25.790080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] The token cache is being invalidate because project 4cd4d5a63dba4a98a8d496e70f3c34fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:25.790367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:25.791064 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fc5324e7-2643-4903-a923-0d694f5a3744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:25.792684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1220/2399] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:25 2022] DELETE /identity/v3/projects/4cd4d5a63dba4a98a8d496e70f3c34fe => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:25.800850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:25.801654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:25.802269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:25.832662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:25.834081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:25.904127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2400] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:25.915003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:25.915448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:25.915873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:25.946908 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:25.948228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:25.957689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1221/2401] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:25.962426 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:25.975215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81faee-2195-4651-89a9-10bf0812511a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:25.983791 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:25.985509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:25.986083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:25.986602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:25.987516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:27:25.987959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:25.990305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.002109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:26.010345 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2402] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:27:26.013003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.023544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fd706-5b08-40a4-80c7-dfff5548aa78 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.028766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.029419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.029688 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.029966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.030641 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.033693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.086419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1222/2403] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:26 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:26.090566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.105391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97925b97-f900-423c-bcbe-335cfbd424e5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.210216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.210998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.211290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.211602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.212295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.214637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.258659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2404] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:26 2022] POST /identity/v3/users => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:26.259262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.271867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1caaca-d7dc-4179-81be-77aa230c1077 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.279006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.279704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.279998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.280286 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.281008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.284224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.309620 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1223/2405] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:26.314653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.326415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112c4e2-5a70-4e3d-84e7-8091cf90e264 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.334566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86947e-bca5-4cec-a158-01bcdfd4c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.336197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86947e-bca5-4cec-a158-01bcdfd4c787 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.336605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86947e-bca5-4cec-a158-01bcdfd4c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.337036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86947e-bca5-4cec-a158-01bcdfd4c787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.337880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86947e-bca5-4cec-a158-01bcdfd4c787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.339867 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86947e-bca5-4cec-a158-01bcdfd4c787 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.357665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2406] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:26.362801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.381295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5b1e-3823-42f6-9d76-ba3fc0b60a28 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.387523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.388364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.388658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.388937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] PATH_INFO: `/v3/projects/04c19bde94e040cc8714495361f47eee/users/ab3cfcd1a84543038a6831309a4000fd/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.408894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c19bde94e040cc8714495361f47eee, user_id=ab3cfcd1a84543038a6831309a4000fd, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.411981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.431789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1224/2407] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:26 2022] PUT /identity/v3/projects/04c19bde94e040cc8714495361f47eee/users/ab3cfcd1a84543038a6831309a4000fd/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:26.444334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.444752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.445204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.480379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] MFA Rules not processed for user `ab3cfcd1a84543038a6831309a4000fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:26.481875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:26.591392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2408] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:26.600733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.768141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:26.801744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc720b-9b14-4b2b-a6a3-e832ae921d89 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.810352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.811527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.812082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.812575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.816191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.818999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:26.862058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:26.862605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1225/2409] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:26 2022] POST /identity/v3/projects => generated 405 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:26.879566 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc39ca-aae8-47b4-8f49-b294ee7617de None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:26.993068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:26.994371 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:26.994845 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:26.995259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:26.996298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:26.998831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:27.020577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:27.048388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2410] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:26 2022] POST /identity/v3/users => generated 309 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:27.052546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:27.064823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338b1d6-8847-415c-93e5-a562fdf817bd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:27.070366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:27.071127 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.071661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.071878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.073553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:27.074849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:27.114205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1226/2411] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:27.118624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:27.138934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7511d-e1d3-4820-8a6b-6a26176ae47f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:27.155219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8494f-7384-4072-a6ff-b6aa2bd0f7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:27.155604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8494f-7384-4072-a6ff-b6aa2bd0f7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.155898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8494f-7384-4072-a6ff-b6aa2bd0f7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.156188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8494f-7384-4072-a6ff-b6aa2bd0f7f7 admin admin] PATH_INFO: `/v3/projects/c6d1f46590b24c1c81d7038b62b49fc7/users/b854b11716304c15b9f8e25c6289a8b0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.195244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8494f-7384-4072-a6ff-b6aa2bd0f7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d1f46590b24c1c81d7038b62b49fc7, user_id=b854b11716304c15b9f8e25c6289a8b0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:27.201081 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8494f-7384-4072-a6ff-b6aa2bd0f7f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:27.231975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2412] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:27 2022] PUT /identity/v3/projects/c6d1f46590b24c1c81d7038b62b49fc7/users/b854b11716304c15b9f8e25c6289a8b0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:27.233531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:27.245733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63358d-2413-4170-9ebc-18a900751fb8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:27.281562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183d437-242e-4933-82ce-055b1810538a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.282263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183d437-242e-4933-82ce-055b1810538a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.282689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183d437-242e-4933-82ce-055b1810538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.318100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d183d437-242e-4933-82ce-055b1810538a None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:27.319603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d183d437-242e-4933-82ce-055b1810538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:27.392547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2413] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:27:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:27.411179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183d437-242e-4933-82ce-055b1810538a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:27.413787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d183d437-242e-4933-82ce-055b1810538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:27.482057 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:27.482901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.483380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.483657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.484496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:27.486810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:27.534402 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1227/2414] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:27.535700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:27.551331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d7cd-3898-4ab1-a4bd-7022f4a8b8bf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:27.563641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183d437-242e-4933-82ce-055b1810538a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:27.569868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e76c3-b789-4549-9fe7-50b17e945e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:27.570861 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e76c3-b789-4549-9fe7-50b17e945e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.571184 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e76c3-b789-4549-9fe7-50b17e945e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.571680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e76c3-b789-4549-9fe7-50b17e945e01 admin admin] PATH_INFO: `/v3/projects/c6d1f46590b24c1c81d7038b62b49fc7/users/b854b11716304c15b9f8e25c6289a8b0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.573812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:27.574071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e76c3-b789-4549-9fe7-50b17e945e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d1f46590b24c1c81d7038b62b49fc7, user_id=b854b11716304c15b9f8e25c6289a8b0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:27.574703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.575173 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.576305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.579341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e76c3-b789-4549-9fe7-50b17e945e01 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:27.612967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:27.626652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1228/2415] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:27 2022] PUT /identity/v3/projects/c6d1f46590b24c1c81d7038b62b49fc7/users/b854b11716304c15b9f8e25c6289a8b0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:27.626866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:27.629920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:27.630386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:27.630757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:27.633934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2416] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:27:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:27:27.700619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] MFA Rules not processed for user `b854b11716304c15b9f8e25c6289a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:27.705024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:27.849162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1229/2417] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:27 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:27.862534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:27.864286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:28.028126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:28.083304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06557af-efc2-4cea-aecb-f286d21a8a57 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:28.216239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365749e7-684e-4c9f-9bac-b966867feaeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:28.217103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365749e7-684e-4c9f-9bac-b966867feaeb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:28.217520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365749e7-684e-4c9f-9bac-b966867feaeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:28.217911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365749e7-684e-4c9f-9bac-b966867feaeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:28.235482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365749e7-684e-4c9f-9bac-b966867feaeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:28.237574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365749e7-684e-4c9f-9bac-b966867feaeb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:28.259866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2418] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:27 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:27:28.483609 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:28.619314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:28.664939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e116ba-43b6-45b6-8cf3-1968242e9370 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:28.678180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:28.680063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:28.680633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:28.680927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] PATH_INFO: `/v3/users/ab3cfcd1a84543038a6831309a4000fd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:28.682680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3cfcd1a84543038a6831309a4000fd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:28.684888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:28.764635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3cfcd1a84543038a6831309a4000fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:28.777625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3cfcd1a84543038a6831309a4000fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:28.778799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3cfcd1a84543038a6831309a4000fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:28.780416 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3cfcd1a84543038a6831309a4000fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:28.806931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3cfcd1a84543038a6831309a4000fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:28.828580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1230/2419] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:28 2022] DELETE /identity/v3/users/ab3cfcd1a84543038a6831309a4000fd => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:28.834183 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:28.834481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:28.834776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:28.857286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:28.858547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:28.989143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2420] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:28.999594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:29.001673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:29.115600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae08e-db2b-4a9f-bc1f-208657a08745 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:29.141541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:29.142645 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:29.143304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:29.143713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:29.163474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:29.166319 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:29.203849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1231/2421] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:30.006223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:30.008939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:30.056613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aea6a-f528-4eeb-85ca-2569f38a6fdc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:30.077434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:30.078419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:30.079475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:30.079916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] PATH_INFO: `/v3/projects/04c19bde94e040cc8714495361f47eee` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:30.081729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c19bde94e040cc8714495361f47eee)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:30.085406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:30.237127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c19bde94e040cc8714495361f47eee', 'request_id': 'req-27665bb0-284d-44e2-a96f-36b853f791f2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.248765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c19bde94e040cc8714495361f47eee', 'request_id': 'req-27665bb0-284d-44e2-a96f-36b853f791f2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.257459 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c19bde94e040cc8714495361f47eee', 'request_id': 'req-27665bb0-284d-44e2-a96f-36b853f791f2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.287472 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] The token cache is being invalidate because project 04c19bde94e040cc8714495361f47eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:30.287771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.288670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.293174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2422] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:30 2022] DELETE /identity/v3/projects/04c19bde94e040cc8714495361f47eee => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:30.294460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:30.308708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b1b7d-12dd-439e-b449-4458c56f869a service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:30.426927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:30.460284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:30.460982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:30.461151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:30.461439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] PATH_INFO: `/v3/users/b854b11716304c15b9f8e25c6289a8b0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:30.462819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b854b11716304c15b9f8e25c6289a8b0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:30.464854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:30.589594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b854b11716304c15b9f8e25c6289a8b0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.619720 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b854b11716304c15b9f8e25c6289a8b0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.620624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b854b11716304c15b9f8e25c6289a8b0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.621388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b854b11716304c15b9f8e25c6289a8b0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.640534 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c72cd8e-678c-4866-92b1-d5e079fe470b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b854b11716304c15b9f8e25c6289a8b0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:30.701109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1232/2423] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:30 2022] DELETE /identity/v3/users/b854b11716304c15b9f8e25c6289a8b0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:31.405394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:31.407762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:31.430783 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27665bb0-284d-44e2-a96f-36b853f791f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:31.471443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:31.562444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:31.564385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:31.564963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:31.565549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] PATH_INFO: `/v3/projects/c6d1f46590b24c1c81d7038b62b49fc7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:31.569033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d1f46590b24c1c81d7038b62b49fc7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:31.573124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:31.685429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d1f46590b24c1c81d7038b62b49fc7', 'request_id': 'req-d8ad0a59-8f52-43bc-9393-f7437af87a62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:31.719470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d1f46590b24c1c81d7038b62b49fc7', 'request_id': 'req-d8ad0a59-8f52-43bc-9393-f7437af87a62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:31.744377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d1f46590b24c1c81d7038b62b49fc7', 'request_id': 'req-d8ad0a59-8f52-43bc-9393-f7437af87a62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:31.814685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] The token cache is being invalidate because project c6d1f46590b24c1c81d7038b62b49fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:31.815283 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:31.816587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8ad0a59-8f52-43bc-9393-f7437af87a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:31.818930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2424] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:31 2022] DELETE /identity/v3/projects/c6d1f46590b24c1c81d7038b62b49fc7 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:31.826210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:31.827108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:31.827108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:31.851476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:31.853170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:32.069633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1233/2425] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:32.087218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.087218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.087218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.157553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:32.158951 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:32.191228 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2426] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:32.198960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:32.219837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7308b6-af51-471d-8db1-26796f93a47f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:32.229082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:32.231143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.232061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.238372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.240655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:27:32.241625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:32.246136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:32.281500 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1234/2427] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:27:32.283777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:32.308679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c081-0c2f-4037-ba20-207edca59351 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:32.318933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:32.320615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.321129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.321629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.323055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:32.326906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:32.395433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2428] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:32 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:27:32.396144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:32.412427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a216ec-d47a-4d45-b631-2e4ec18dec17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:32.539995 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:32.541377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.541901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.542412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.543781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:32.548074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:32.560501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:32.596363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1235/2429] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:32 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:32.599385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:32.622492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013b98f-3c05-43a2-b7c5-71784a7f2c91 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:32.635111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:32.636266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.636903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.637449 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.638868 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:32.642760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:32.674926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2430] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:32.675178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:32.691213 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9e8bd-c7b8-4d2e-bceb-3544351f5cb7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:32.699957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b036b-16ab-420c-af42-091f9165c479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:32.701234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b036b-16ab-420c-af42-091f9165c479 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.701830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b036b-16ab-420c-af42-091f9165c479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.702410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b036b-16ab-420c-af42-091f9165c479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.704326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b036b-16ab-420c-af42-091f9165c479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:32.708939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b036b-16ab-420c-af42-091f9165c479 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:32.739229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1236/2431] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:32.741658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:32.768781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e08230-7a3a-4834-812c-c3676be7714e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:32.786495 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:32.787392 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.787794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.788255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] PATH_INFO: `/v3/projects/382f366b6db94576be56c4f24154b68f/users/7bbedb171b084448b1b79851ca0614da/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.830318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=382f366b6db94576be56c4f24154b68f, user_id=7bbedb171b084448b1b79851ca0614da, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:32.836375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:32.895377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2432] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:32 2022] PUT /identity/v3/projects/382f366b6db94576be56c4f24154b68f/users/7bbedb171b084448b1b79851ca0614da/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:32.897740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:32.898518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:32.898663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:32.996273 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] MFA Rules not processed for user `7bbedb171b084448b1b79851ca0614da`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:32.999107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:33.175203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1237/2433] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:33.182017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:33.313273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:33.369492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dc0b6-c809-4193-96e1-81b7cfee0e0b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:33.382413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:33.384065 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:33.384603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:33.385301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:33.386667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:33.390227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:33.450020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2434] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:33 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:33.450865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:33.480202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c41cd4-c16f-4208-9d45-a066cd2766fa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:33.598006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:33.598686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:33.600031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:33.602821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:33.602821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:33.604084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:33.648962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1238/2435] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:33 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:33.655289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:33.677130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7020d8e-2f9e-4e49-b212-a75af072dfbb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:33.683073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:33.684271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:33.684574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:33.684862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:33.685927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:33.703835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:33.727849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2436] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:33.729663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:33.760800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb16cd-e66b-4f91-99af-21c6c962fb71 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:33.770803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945b13e-bfbf-4762-8032-ed2639be5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:33.772171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b13e-bfbf-4762-8032-ed2639be5dea admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:33.772881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b13e-bfbf-4762-8032-ed2639be5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:33.773571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b13e-bfbf-4762-8032-ed2639be5dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:33.775151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945b13e-bfbf-4762-8032-ed2639be5dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:33.779216 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945b13e-bfbf-4762-8032-ed2639be5dea admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:33.806195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1239/2437] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:33.810510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:33.822741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b6fdb-36a9-4045-8f72-c6e060488080 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:33.829192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:33.830368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:33.830736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:33.831100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] PATH_INFO: `/v3/projects/10d7bcdcde5649d79a186831d68f013a/users/ad4fa57e18a649519d59adf6eb15713a/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:33.853161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d7bcdcde5649d79a186831d68f013a, user_id=ad4fa57e18a649519d59adf6eb15713a, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:33.856106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:33.877363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2438] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:33 2022] PUT /identity/v3/projects/10d7bcdcde5649d79a186831d68f013a/users/ad4fa57e18a649519d59adf6eb15713a/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:33.889540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:33.889892 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:33.890265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:33.940079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] MFA Rules not processed for user `ad4fa57e18a649519d59adf6eb15713a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:33.941392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:33.993530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:34.041674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1240/2439] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:34.042967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:34.177553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a1110-0aea-432b-bda0-3ae6ec136005 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:34.185882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:34.186986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.187592 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.188103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.189593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:34.193478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:34.257704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2440] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:34 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:27:34.258911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:34.272615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cd8d9-eeeb-47a8-a12d-0d93034127fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:34.374134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:34.375086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.376130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.376530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.377351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:34.379758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:34.419087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1241/2441] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:34 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:34.424613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:34.449669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0433a7-beeb-4a5a-87b7-978e36eb34f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:34.458300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:34.459893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.460485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.460960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.462553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:34.465491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:34.480287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2442] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:34.489649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:34.510746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d430-293d-4843-aedc-7255dd94bd97 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:34.519654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:34.521094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.521739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.522292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] PATH_INFO: `/v3/projects/beb460c4a2f945968ccc00b8997fa0ef/users/725fd8a2fbac47b9a64ec3f37b1bb424/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.555630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb460c4a2f945968ccc00b8997fa0ef, user_id=725fd8a2fbac47b9a64ec3f37b1bb424, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:34.561289 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:34.585587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1242/2443] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:34 2022] PUT /identity/v3/projects/beb460c4a2f945968ccc00b8997fa0ef/users/725fd8a2fbac47b9a64ec3f37b1bb424/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:34.591436 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:34.608541 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d47d6-b732-4b81-8901-d096931e1371 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:34.712927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7f05a-29c7-43b6-8578-398f50c59279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:34.714421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f05a-29c7-43b6-8578-398f50c59279 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.714918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f05a-29c7-43b6-8578-398f50c59279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.715424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f05a-29c7-43b6-8578-398f50c59279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.716386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7f05a-29c7-43b6-8578-398f50c59279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:34.718291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7f05a-29c7-43b6-8578-398f50c59279 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:34.732798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2444] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:34.737011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:34.754389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70f5d-ae9f-4fbf-890d-3be972771982 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:34.760607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:34.763512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.763960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.764346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] PATH_INFO: `/v3/projects/beb460c4a2f945968ccc00b8997fa0ef/users/725fd8a2fbac47b9a64ec3f37b1bb424/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.767586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb460c4a2f945968ccc00b8997fa0ef, user_id=725fd8a2fbac47b9a64ec3f37b1bb424, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:34.770548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:34.789315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1243/2445] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:34 2022] PUT /identity/v3/projects/beb460c4a2f945968ccc00b8997fa0ef/users/725fd8a2fbac47b9a64ec3f37b1bb424/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:34.800641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:34.800952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:34.801340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:34.834016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:34.872591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] MFA Rules not processed for user `725fd8a2fbac47b9a64ec3f37b1bb424`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:34.877789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:35.053991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2446] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:35.070601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:35.073197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:35.161792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58929c4-c218-42c6-aeab-22df15b1c8a7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:35.176923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:35.177740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:35.178028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:35.178581 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:35.191893 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:35.193392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:35.211584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1244/2447] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:27:36.101182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:36.105378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:36.132844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3beb6-3d1c-4d16-bc29-72d0d214cf1a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:36.140977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:36.142529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:36.143455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:36.144225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:36.257084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:36.258635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:36.270240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2448] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:41.150407 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:41.160805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01398-70b0-4961-921a-b2f7a0d7169f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:41.164532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b51eb-56ca-4a9d-8f15-a4f2ca6c8fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:41.165155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b51eb-56ca-4a9d-8f15-a4f2ca6c8fcb service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:41.165441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b51eb-56ca-4a9d-8f15-a4f2ca6c8fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:41.165809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b51eb-56ca-4a9d-8f15-a4f2ca6c8fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:41.199862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:41.236197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b51eb-56ca-4a9d-8f15-a4f2ca6c8fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:41.237721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b51eb-56ca-4a9d-8f15-a4f2ca6c8fcb service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:41.250501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1245/2449] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:43.661403 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:43.766514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b7f7-d193-42b1-bf55-faf0a45b9330 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:43.772060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:43.772802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:43.773090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:43.773376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] PATH_INFO: `/v3/users/7bbedb171b084448b1b79851ca0614da` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:43.774685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bbedb171b084448b1b79851ca0614da)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:43.777502 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:43.833616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:43.856042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bbedb171b084448b1b79851ca0614da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:43.857203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bbedb171b084448b1b79851ca0614da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:43.871570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bbedb171b084448b1b79851ca0614da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:43.872956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bbedb171b084448b1b79851ca0614da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:43.889718 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bbedb171b084448b1b79851ca0614da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:43.909430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2450] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:43 2022] DELETE /identity/v3/users/7bbedb171b084448b1b79851ca0614da => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:43.915081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:43.916379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:43.917116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:43.941716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:43.943139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:44.019705 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1246/2451] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:44.033080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:44.034599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:44.103189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831db808-b6eb-49bb-a448-f4cf9cd4f0b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:44.127570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:44.128562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:44.129007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:44.129442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:44.151051 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:44.153420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:44.174821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2452] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:27:44.612803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:44.614788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:44.633675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dc5fe-bac3-4831-97f2-f4f4da12a3f2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:44.642198 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:44.643223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:44.644386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:44.644674 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] PATH_INFO: `/v3/projects/382f366b6db94576be56c4f24154b68f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:44.649080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=382f366b6db94576be56c4f24154b68f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:44.651964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:44.752900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '382f366b6db94576be56c4f24154b68f', 'request_id': 'req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.769244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '382f366b6db94576be56c4f24154b68f', 'request_id': 'req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.782513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '382f366b6db94576be56c4f24154b68f', 'request_id': 'req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.798548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] The token cache is being invalidate because project 382f366b6db94576be56c4f24154b68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:44.798912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.799878 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.802896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1247/2453] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:44 2022] DELETE /identity/v3/projects/382f366b6db94576be56c4f24154b68f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:44.808831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:44.822172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae378a6-a800-4582-8d26-6c64b8487e38 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:44.903438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:44.905206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:44.905862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:44.906161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] PATH_INFO: `/v3/users/ad4fa57e18a649519d59adf6eb15713a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:44.907809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4fa57e18a649519d59adf6eb15713a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:44.910223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:44.988625 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4fa57e18a649519d59adf6eb15713a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.989389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4fa57e18a649519d59adf6eb15713a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:44.996698 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:45.002125 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4fa57e18a649519d59adf6eb15713a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.004299 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4fa57e18a649519d59adf6eb15713a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.026779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4fa57e18a649519d59adf6eb15713a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.052853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2454] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:44 2022] DELETE /identity/v3/users/ad4fa57e18a649519d59adf6eb15713a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:45.640795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:45.642847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:45.690362 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb997fa5-7d52-45cb-b181-c3c6a416a6ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:45.778494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:45.801328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:45.803423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:45.804190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:45.804785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] PATH_INFO: `/v3/projects/10d7bcdcde5649d79a186831d68f013a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:45.807524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d7bcdcde5649d79a186831d68f013a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:45.812423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:45.929492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d7bcdcde5649d79a186831d68f013a', 'request_id': 'req-03239705-e32c-4efb-b9e8-23084dc2ec1b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.944389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d7bcdcde5649d79a186831d68f013a', 'request_id': 'req-03239705-e32c-4efb-b9e8-23084dc2ec1b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.956698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d7bcdcde5649d79a186831d68f013a', 'request_id': 'req-03239705-e32c-4efb-b9e8-23084dc2ec1b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.983254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] The token cache is being invalidate because project 10d7bcdcde5649d79a186831d68f013a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:45.983801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.985085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:45.987295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1248/2455] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:45 2022] DELETE /identity/v3/projects/10d7bcdcde5649d79a186831d68f013a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:45.993792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:45.996559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:46.016747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f743-83b0-47fe-bd6e-de0539b869d7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:46.125615 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:46.126691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:46.127205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:46.127727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] PATH_INFO: `/v3/users/725fd8a2fbac47b9a64ec3f37b1bb424` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:46.129892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] RBAC: Authorizing `identity:delete_user(user_id=725fd8a2fbac47b9a64ec3f37b1bb424)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:46.133545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:46.239433 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725fd8a2fbac47b9a64ec3f37b1bb424'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:46.240572 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725fd8a2fbac47b9a64ec3f37b1bb424'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:46.252537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725fd8a2fbac47b9a64ec3f37b1bb424'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:46.254028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725fd8a2fbac47b9a64ec3f37b1bb424'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:46.284949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e7516c0-40cc-4e83-8a6b-01a6fca138df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725fd8a2fbac47b9a64ec3f37b1bb424'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:46.309691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2456] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:45 2022] DELETE /identity/v3/users/725fd8a2fbac47b9a64ec3f37b1bb424 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:46.754937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:46.756549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:46.782668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239705-e32c-4efb-b9e8-23084dc2ec1b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:46.890636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:46.892258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:46.892597 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:46.892904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] PATH_INFO: `/v3/projects/beb460c4a2f945968ccc00b8997fa0ef` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:46.894268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb460c4a2f945968ccc00b8997fa0ef)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:46.897182 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:46.979940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb460c4a2f945968ccc00b8997fa0ef', 'request_id': 'req-4917041c-a11d-4976-bee9-4ee35b2ee060'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:46.992241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb460c4a2f945968ccc00b8997fa0ef', 'request_id': 'req-4917041c-a11d-4976-bee9-4ee35b2ee060'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:47.000146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb460c4a2f945968ccc00b8997fa0ef', 'request_id': 'req-4917041c-a11d-4976-bee9-4ee35b2ee060'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:47.013024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] The token cache is being invalidate because project beb460c4a2f945968ccc00b8997fa0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:27:47.013348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:47.014049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4917041c-a11d-4976-bee9-4ee35b2ee060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:47.016221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1249/2457] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:46 2022] DELETE /identity/v3/projects/beb460c4a2f945968ccc00b8997fa0ef => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:47.023171 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.023862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.023862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.046924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:47.048229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:47.104335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:47.126512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2458] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:47.135522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.135812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.136099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.160742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:47.162287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:47.171683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1250/2459] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:47.176840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.192724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c6ac-0858-49de-857d-851864d50d5f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.198042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.198684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.198965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.199240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.199995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:27:47.200295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.201964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.215202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2460] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:27:47.219900 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.229706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34436a02-bb6d-47d9-943d-f13dd34d9217 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.235506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.236051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.236336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.236612 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.237300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.239239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.244476 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:47.276241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1251/2461] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:27:47.282202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.293224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fabf9-3c0c-4692-82e9-f7af42bc31e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.371285 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.371968 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.372252 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.372531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.373226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.376240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.403727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2462] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/users => generated 305 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:47.408817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.421815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085cd0-8ad6-4765-ac19-9bc1caf7a761 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.427465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.428130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.428424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.428708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.429432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.431090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.442938 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1252/2463] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:47.447617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.461369 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc55e25-cb9a-42e0-ab14-61caa156cf8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.466334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557b39a-edef-4946-aadc-5880e274a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.466978 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557b39a-edef-4946-aadc-5880e274a77d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.467260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557b39a-edef-4946-aadc-5880e274a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.467569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557b39a-edef-4946-aadc-5880e274a77d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.468296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557b39a-edef-4946-aadc-5880e274a77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.470148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557b39a-edef-4946-aadc-5880e274a77d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.482275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2464] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:47.487685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.504147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5546af1-8fef-4bf2-b72f-0cb198c5f2b4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.511987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.513149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.513662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.514076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] PATH_INFO: `/v3/projects/e9b5c532b60a4c1daae8df63bde63f3e/users/b87a628ef5954637a70b4324e365936d/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.543621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b5c532b60a4c1daae8df63bde63f3e, user_id=b87a628ef5954637a70b4324e365936d, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.550089 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.574419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1253/2465] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:47 2022] PUT /identity/v3/projects/e9b5c532b60a4c1daae8df63bde63f3e/users/b87a628ef5954637a70b4324e365936d/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:47.648195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.648473 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.648742 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.683587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] MFA Rules not processed for user `b87a628ef5954637a70b4324e365936d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:47.684981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:47.786663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2466] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:27:47.792186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.916873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcdab5-0f65-413d-aee1-3befe7076714 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:47.925316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:47.926105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:47.926482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:47.926868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:47.927694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:47.929978 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:47.965414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1254/2467] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/projects => generated 409 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:27:47.969852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:47.991444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead5b37-35f0-4652-8e55-289e54589b50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.063457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:48.089386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.090497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.090996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.091530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.092787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:48.097424 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:48.126076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2468] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:27:47 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:27:48.131915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:48.148584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f594f-171d-452d-8090-9974852668dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.154610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.156075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.156437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.156989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.158035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:48.160349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:48.182915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1255/2469] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:27:48.188735 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:48.211293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99626b2c-b345-470a-b964-8205d049fc33 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.220387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.221198 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.221627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.221924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] PATH_INFO: `/v3/projects/7135e10790db4b548fbdc2299214136a/users/146c290f775c447ea8e20183b2cce880/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.241486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7135e10790db4b548fbdc2299214136a, user_id=146c290f775c447ea8e20183b2cce880, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:48.244244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:48.261943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2470] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:48 2022] PUT /identity/v3/projects/7135e10790db4b548fbdc2299214136a/users/146c290f775c447ea8e20183b2cce880/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:48.266601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:48.277887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f516d9-5659-4b3f-97db-5cb25567d000 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.376266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba220fd8-c4d5-40f1-a4a2-3fabc082447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.377927 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba220fd8-c4d5-40f1-a4a2-3fabc082447c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.378432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba220fd8-c4d5-40f1-a4a2-3fabc082447c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.378929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba220fd8-c4d5-40f1-a4a2-3fabc082447c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.379871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba220fd8-c4d5-40f1-a4a2-3fabc082447c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:48.381693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba220fd8-c4d5-40f1-a4a2-3fabc082447c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:48.394666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1256/2471] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:27:48.399526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:48.414546 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558803f5-b2ea-4b30-96a9-ff304ae4f7dc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.421362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.422837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.423294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.423917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] PATH_INFO: `/v3/projects/7135e10790db4b548fbdc2299214136a/users/146c290f775c447ea8e20183b2cce880/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.426549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7135e10790db4b548fbdc2299214136a, user_id=146c290f775c447ea8e20183b2cce880, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:48.429854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:48.442434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2472] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:27:48 2022] PUT /identity/v3/projects/7135e10790db4b548fbdc2299214136a/users/146c290f775c447ea8e20183b2cce880/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:27:48.454280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.454575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.454860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.486163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] MFA Rules not processed for user `146c290f775c447ea8e20183b2cce880`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:48.487471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:48.499826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:48.600443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1257/2473] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:48 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:48.613199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:48.615213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:48.695152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67973fde-a372-4831-8228-626aa382ef8a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.699745 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c463d-3a3e-454a-8b12-25bb5256a231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.700412 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c463d-3a3e-454a-8b12-25bb5256a231 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.700702 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c463d-3a3e-454a-8b12-25bb5256a231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.700990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c463d-3a3e-454a-8b12-25bb5256a231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:48.712346 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c463d-3a3e-454a-8b12-25bb5256a231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:48.714218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c463d-3a3e-454a-8b12-25bb5256a231 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:48.726728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2474] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:27:48.951574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:48.971246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bbbce-acb7-48b4-8309-745210bb7337 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:48.977006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:48.977750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:48.978242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:48.978727 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:49.071456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:49.074214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:49.096308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1258/2475] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:27:50.146318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:50.146946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:50.146946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:50.169092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:50.170347 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:50.243429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2476] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 05:27:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:50.257826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:50.259913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:50.415859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52833f8-3db3-434f-bbcb-4513883ae7e1 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:50.428654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d4016-9cdd-4f52-ad3a-f279756f6edd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:50.429134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4016-9cdd-4f52-ad3a-f279756f6edd service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:50.429420 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4016-9cdd-4f52-ad3a-f279756f6edd service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:50.429707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4016-9cdd-4f52-ad3a-f279756f6edd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:50.445527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4016-9cdd-4f52-ad3a-f279756f6edd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:50.447164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4016-9cdd-4f52-ad3a-f279756f6edd service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:50.457627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1259/2477] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:27:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 05:27:58.698504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:58.763525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:58.784284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dae28-c643-471c-96f7-7ded26e47833 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:58.789794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:58.790483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:58.790767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:58.791047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] PATH_INFO: `/v3/users/b87a628ef5954637a70b4324e365936d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:58.792853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87a628ef5954637a70b4324e365936d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:58.794988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:58.905587 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87a628ef5954637a70b4324e365936d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:58.908447 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87a628ef5954637a70b4324e365936d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:58.944063 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87a628ef5954637a70b4324e365936d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:58.945950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87a628ef5954637a70b4324e365936d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:58.968680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87a628ef5954637a70b4324e365936d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:27:58.997606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2478] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:27:58 2022] DELETE /identity/v3/users/b87a628ef5954637a70b4324e365936d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:27:59.002789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:59.003748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:59.004027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:59.040383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:27:59.052239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:59.143535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:27:59.228036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1260/2479] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:27:59.244989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:59.247069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:59.359521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcad6b0-fec4-43d5-928f-d84db756498f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:59.364024 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:59.364682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:59.364972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:59.365258 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:59.377813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:59.379286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:59.391231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2480] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:27:59.886401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:27:59.888152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:27:59.910058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416dc29-d74a-4472-9095-1b5f6c498e7b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:27:59.931472 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:27:59.932544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:27:59.932943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:27:59.933338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] PATH_INFO: `/v3/projects/e9b5c532b60a4c1daae8df63bde63f3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:27:59.934826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b5c532b60a4c1daae8df63bde63f3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:27:59.938367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:27:59.995316 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b5c532b60a4c1daae8df63bde63f3e', 'request_id': 'req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.004328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b5c532b60a4c1daae8df63bde63f3e', 'request_id': 'req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.011672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b5c532b60a4c1daae8df63bde63f3e', 'request_id': 'req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.024585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] The token cache is being invalidate because project e9b5c532b60a4c1daae8df63bde63f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:00.025207 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.025944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.027668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1261/2481] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:27:59 2022] DELETE /identity/v3/projects/e9b5c532b60a4c1daae8df63bde63f3e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:00.032372 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:00.043057 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9e915-d003-444b-b706-159782b53247 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:00.127115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:00.127838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:00.128135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:00.128420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] PATH_INFO: `/v3/users/146c290f775c447ea8e20183b2cce880` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:00.129800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146c290f775c447ea8e20183b2cce880)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:00.131946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:00.147830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:00.231886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146c290f775c447ea8e20183b2cce880'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.232774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146c290f775c447ea8e20183b2cce880'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.253781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146c290f775c447ea8e20183b2cce880'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.255151 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146c290f775c447ea8e20183b2cce880'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.287263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfdbbd12-a884-435a-bfff-0d2e403752a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146c290f775c447ea8e20183b2cce880'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:00.305411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2482] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:00 2022] DELETE /identity/v3/users/146c290f775c447ea8e20183b2cce880 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:00.818564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:00.820242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:00.834841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be16839-4fb3-46b7-929e-7ea60c9d5c6f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:00.909155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:00.910468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:00.910760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:00.911068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] PATH_INFO: `/v3/projects/7135e10790db4b548fbdc2299214136a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:00.912382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7135e10790db4b548fbdc2299214136a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:00.914361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:00.995989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7135e10790db4b548fbdc2299214136a', 'request_id': 'req-92185faa-d3ab-44f4-8c56-56c54629e071'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:01.009033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7135e10790db4b548fbdc2299214136a', 'request_id': 'req-92185faa-d3ab-44f4-8c56-56c54629e071'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:01.012773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:01.020256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7135e10790db4b548fbdc2299214136a', 'request_id': 'req-92185faa-d3ab-44f4-8c56-56c54629e071'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:01.036491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] The token cache is being invalidate because project 7135e10790db4b548fbdc2299214136a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:01.036786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:01.037510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-92185faa-d3ab-44f4-8c56-56c54629e071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:01.039129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1262/2483] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:00 2022] DELETE /identity/v3/projects/7135e10790db4b548fbdc2299214136a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:01.044665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.045960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.046129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.068513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:01.069903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:01.148196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2484] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:01.163923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.164270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.164615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.194952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:01.197170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:01.211941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1263/2485] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:01.216537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.227725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35606d2e-1720-4d1d-9688-b38b83ff45c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.233600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.233600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.234036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.234197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.234895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:01.235209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.236922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.252105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2486] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:28:01.257179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.269191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3e98e-21da-4588-a696-ee3a3c0d702d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.275483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.276215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.276560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.276950 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.277702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.280566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.320706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1264/2487] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:01.325246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.338010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e608803-121d-4a73-aff8-c0e806ed9fb9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.404092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.405310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.405600 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.405882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.406937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.408571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.439933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2488] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:01.441046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.452945 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1d23b-38f8-4b97-be99-b39e3d478865 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.457812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.458447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.458726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.458999 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.459726 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.461346 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.474208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1265/2489] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:01.478251 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.485579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:01.489906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb44b-9b4a-49b5-9306-119b620d388c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.496764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cdb13-b0d4-41b4-a19f-583c74887412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.497374 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cdb13-b0d4-41b4-a19f-583c74887412 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.497871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cdb13-b0d4-41b4-a19f-583c74887412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.498543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cdb13-b0d4-41b4-a19f-583c74887412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.499407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cdb13-b0d4-41b4-a19f-583c74887412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.501630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cdb13-b0d4-41b4-a19f-583c74887412 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.516613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2490] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:01.521692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.548314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630a45-d3af-448b-a031-336649131677 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.555745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.556569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.557176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.557176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] PATH_INFO: `/v3/projects/804260d3eb2c4aeab5fe0d90a4cd3d08/users/cc054bdb1d5d4c52982493739080b36e/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.578933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804260d3eb2c4aeab5fe0d90a4cd3d08, user_id=cc054bdb1d5d4c52982493739080b36e, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.581884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.608623 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1266/2491] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:01 2022] PUT /identity/v3/projects/804260d3eb2c4aeab5fe0d90a4cd3d08/users/cc054bdb1d5d4c52982493739080b36e/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:01.620427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.620671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.620882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.658644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] MFA Rules not processed for user `cc054bdb1d5d4c52982493739080b36e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:01.660114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:01.765669 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2492] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:01.767097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.857218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235534a-bc80-451a-b74e-a68870fa3180 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:01.863332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:01.864169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:01.864462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:01.864744 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:01.865693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:01.868281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:01.928378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1267/2493] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:01.943518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:01.965904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfc057-8033-41d4-bf56-a2860607c1d9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:02.070033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:02.070033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.070033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.070033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:02.071914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:02.072305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:02.132219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2494] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:01 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:02.146128 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:02.166259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61444895-346c-4817-bfbd-cf3495e9f42f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:02.179329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:02.180557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.181277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.181915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:02.183404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:02.188229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:02.222898 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1268/2495] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:02.226227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:02.267681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03d8c9-af8d-4cd0-a05c-e252f6cd2299 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:02.280825 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:02.281695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.282361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.282733 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] PATH_INFO: `/v3/projects/a50ae93aa66a4cd29018b9dd6a2ca7c7/users/bc291944166644c88337a7832e2995e0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:02.306115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50ae93aa66a4cd29018b9dd6a2ca7c7, user_id=bc291944166644c88337a7832e2995e0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:02.310304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:02.331407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2496] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:02 2022] PUT /identity/v3/projects/a50ae93aa66a4cd29018b9dd6a2ca7c7/users/bc291944166644c88337a7832e2995e0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:02.335610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:02.348240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c19370-0d8f-4475-889c-fa472d24d4e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:02.430468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:02.441571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b52aad-df14-4896-b6bf-d642354d833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:02.442434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52aad-df14-4896-b6bf-d642354d833b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.442886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52aad-df14-4896-b6bf-d642354d833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.443912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52aad-df14-4896-b6bf-d642354d833b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:02.447340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b52aad-df14-4896-b6bf-d642354d833b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:02.456652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b52aad-df14-4896-b6bf-d642354d833b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:02.481779 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1269/2497] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:02.487043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:02.499690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558062-8bd3-45ab-a33f-148e3b9371ff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:02.506114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:02.507039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.507448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.507794 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] PATH_INFO: `/v3/projects/a50ae93aa66a4cd29018b9dd6a2ca7c7/users/bc291944166644c88337a7832e2995e0/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:02.510077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50ae93aa66a4cd29018b9dd6a2ca7c7, user_id=bc291944166644c88337a7832e2995e0, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:02.513221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:02.533109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2498] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:02 2022] PUT /identity/v3/projects/a50ae93aa66a4cd29018b9dd6a2ca7c7/users/bc291944166644c88337a7832e2995e0/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:02.552234 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.552953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.553650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:02.612600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] MFA Rules not processed for user `bc291944166644c88337a7832e2995e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:02.614936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:02.734051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1270/2499] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:02.751602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:02.758528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:02.832789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:02.893239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a842d-27a1-40ab-8060-de144b1d11dd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:02.911079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69bcc2-a416-41a9-9a7d-872b9b8b74c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:02.912481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69bcc2-a416-41a9-9a7d-872b9b8b74c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:02.914729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69bcc2-a416-41a9-9a7d-872b9b8b74c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:02.915537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69bcc2-a416-41a9-9a7d-872b9b8b74c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:03.011293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69bcc2-a416-41a9-9a7d-872b9b8b74c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:03.012901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69bcc2-a416-41a9-9a7d-872b9b8b74c8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:03.034726 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2500] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:28:05.890025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:05.993706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5664c-77cc-4d43-9426-3b0fff1d201c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:06.001459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:06.002506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:06.002981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:06.003582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] PATH_INFO: `/v3/users/cc054bdb1d5d4c52982493739080b36e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:06.005894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc054bdb1d5d4c52982493739080b36e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:06.009453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:06.080830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc054bdb1d5d4c52982493739080b36e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:06.090600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc054bdb1d5d4c52982493739080b36e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:06.091518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc054bdb1d5d4c52982493739080b36e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:06.093932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc054bdb1d5d4c52982493739080b36e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:06.109921 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc054bdb1d5d4c52982493739080b36e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:06.119068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:06.127176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1271/2501] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:05 2022] DELETE /identity/v3/users/cc054bdb1d5d4c52982493739080b36e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:06.132335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926638ea-65b3-49ec-9c13-b67b2e2496d3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:06.132947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926638ea-65b3-49ec-9c13-b67b2e2496d3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:06.132947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926638ea-65b3-49ec-9c13-b67b2e2496d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:06.163006 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-926638ea-65b3-49ec-9c13-b67b2e2496d3 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:06.164437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-926638ea-65b3-49ec-9c13-b67b2e2496d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:06.303585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2502] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:06.313530 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:06.315686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:06.409766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dd51e-21ee-4c77-a9a7-bdd079ed9ce8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:06.420681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:06.421449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:06.421738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:06.422160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:06.435197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:06.437412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:06.451762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1272/2503] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:06.572861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:06.574025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:06.575367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:06.610413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:06.612691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:06.701830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2504] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 05:28:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:06.718649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:06.720355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:06.820592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97753c-9a80-4538-abfb-17bf7502735b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:06.826456 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:06.827114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:06.827434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:06.827713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:06.840334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:06.845028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:06.863845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1273/2505] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:28:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:28:06.887431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:06.889834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:06.908297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2fd0f-c100-4455-9fb8-edde6e3c9b89 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:06.914820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:06.915877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:06.916147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:06.916434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] PATH_INFO: `/v3/projects/804260d3eb2c4aeab5fe0d90a4cd3d08` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:06.917888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804260d3eb2c4aeab5fe0d90a4cd3d08)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:06.921594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:06.979431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:07.006326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804260d3eb2c4aeab5fe0d90a4cd3d08', 'request_id': 'req-86a46cc9-ef08-4980-9ea3-658733fa2c97'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.015043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804260d3eb2c4aeab5fe0d90a4cd3d08', 'request_id': 'req-86a46cc9-ef08-4980-9ea3-658733fa2c97'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.023970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804260d3eb2c4aeab5fe0d90a4cd3d08', 'request_id': 'req-86a46cc9-ef08-4980-9ea3-658733fa2c97'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.046609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] The token cache is being invalidate because project 804260d3eb2c4aeab5fe0d90a4cd3d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:07.047450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.048914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.051878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2506] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:06 2022] DELETE /identity/v3/projects/804260d3eb2c4aeab5fe0d90a4cd3d08 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:07.058200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:07.079706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5fcf9-87f0-4593-be15-ba1699a9ea88 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:07.180357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:07.181029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:07.181318 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:07.182368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] PATH_INFO: `/v3/users/bc291944166644c88337a7832e2995e0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:07.183818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc291944166644c88337a7832e2995e0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:07.186370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:07.237364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:07.255100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc291944166644c88337a7832e2995e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.265980 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc291944166644c88337a7832e2995e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.266723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc291944166644c88337a7832e2995e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.267735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc291944166644c88337a7832e2995e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.284056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc291944166644c88337a7832e2995e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.302437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1274/2507] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:07 2022] DELETE /identity/v3/users/bc291944166644c88337a7832e2995e0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:07.607721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:07.609770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:07.706518 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a46cc9-ef08-4980-9ea3-658733fa2c97 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:07.752967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:07.754844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:07.784359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:07.785712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:07.786047 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:07.786323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:07.788274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:07.868934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671baf0-1549-4d2f-929b-27a31434f3e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:07.874798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:07.875495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:07.875787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:07.876917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] PATH_INFO: `/v3/projects/a50ae93aa66a4cd29018b9dd6a2ca7c7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:07.878187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50ae93aa66a4cd29018b9dd6a2ca7c7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:07.880521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:07.880689 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:07.883574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b4de3-cef3-4a45-a152-297561ee704d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:07.902308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2508] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:07.941191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50ae93aa66a4cd29018b9dd6a2ca7c7', 'request_id': 'req-8bbb65ba-cf79-4a82-9901-880f462f6b49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.952221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50ae93aa66a4cd29018b9dd6a2ca7c7', 'request_id': 'req-8bbb65ba-cf79-4a82-9901-880f462f6b49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.959514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50ae93aa66a4cd29018b9dd6a2ca7c7', 'request_id': 'req-8bbb65ba-cf79-4a82-9901-880f462f6b49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.972369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] The token cache is being invalidate because project a50ae93aa66a4cd29018b9dd6a2ca7c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:07.972702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.973469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bbb65ba-cf79-4a82-9901-880f462f6b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:07.975400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1275/2509] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:07 2022] DELETE /identity/v3/projects/a50ae93aa66a4cd29018b9dd6a2ca7c7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:07.983212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:07.984086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:07.984543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.010225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:08.011910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:08.038965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:08.105190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2510] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:08.115149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.115737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.115920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.146746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:08.148563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:08.163302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1276/2511] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:08.170446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.197876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a7151-18b0-4ac0-a9e7-c0cfe224bf6f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.208316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.209558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.210161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.210762 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.212437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:08.213131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:08.218629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:08.245195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2512] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:28:08.252613 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.278532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4b28-da29-4857-9565-6d06367e6bc7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.288766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.290302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.291056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.291822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.293449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:08.298309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:08.364915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1277/2513] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:08.367555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.380416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839227e-a8a5-470a-95d3-36904ee240a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.464548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.465192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.465482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.465765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.466482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:08.468547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:08.501801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2514] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:08 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:08.505954 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.518493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa64b-49c9-4d6e-94d8-26bcb3a50da5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.528196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.529055 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.529427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.529819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.531502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:08.532630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:08.540153 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:08.548700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1278/2515] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:08.553174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.566927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7d10-4b93-4c2c-b8b0-cf9c670d46b3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.574294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d51359-8917-4b3c-b478-28d8b84dfc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.575819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d51359-8917-4b3c-b478-28d8b84dfc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.577124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d51359-8917-4b3c-b478-28d8b84dfc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.577757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d51359-8917-4b3c-b478-28d8b84dfc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.581811 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d51359-8917-4b3c-b478-28d8b84dfc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:08.585895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d51359-8917-4b3c-b478-28d8b84dfc55 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:08.615415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2516] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:08.618669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.633934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3d710-0975-4a9a-a8ce-4fbd73a2ad63 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.640743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.641457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.641743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.642029 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] PATH_INFO: `/v3/projects/719297a86f0a4930b4daf3b9f663e472/users/5b1d0da1157a46c283bd442ad14d31b5/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.663676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] RBAC: Authorizing `identity:create_grant(project_id=719297a86f0a4930b4daf3b9f663e472, user_id=5b1d0da1157a46c283bd442ad14d31b5, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:08.666926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:08.703503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1279/2517] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:08 2022] PUT /identity/v3/projects/719297a86f0a4930b4daf3b9f663e472/users/5b1d0da1157a46c283bd442ad14d31b5/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:08.706170 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.706896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.707339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:08.732760 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:08.734442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:08.855400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2518] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:08.868300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:08.870465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:08.974582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cc5cb-6975-4bc1-aa06-d567fe95136f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:08.980796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:08.981838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:08.982854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:08.983411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:09.000667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:09.003465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:09.023873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1280/2519] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:13.473695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:13.476260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:13.621906 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:13.803179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:13.806005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:13.850895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d6197-c51b-4a90-97af-2c98ba28382d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:13.860901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e731c24-00ed-429e-9d21-8c42f2b46e6d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:13.872715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:13.873858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:13.877319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:13.884027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:13.893838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:13.896635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:13.897708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:13.899600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:13.900114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] PATH_INFO: `/v3/users/c2db6ada97da4ee89f3ce8de98f2653a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:13.904106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2db6ada97da4ee89f3ce8de98f2653a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:13.921355 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:13.977842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:13.984656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:14.032207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2520] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:28:14.076426 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2db6ada97da4ee89f3ce8de98f2653a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:14.089498 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2db6ada97da4ee89f3ce8de98f2653a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:14.090227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2db6ada97da4ee89f3ce8de98f2653a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:14.091890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2db6ada97da4ee89f3ce8de98f2653a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:14.112313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b013d59c-cb6e-4633-9aa9-4e15384d58e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2db6ada97da4ee89f3ce8de98f2653a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:14.130368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1281/2521] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:13 2022] DELETE /identity/v3/users/c2db6ada97da4ee89f3ce8de98f2653a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:15.151460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:15.182077 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70406b4-f355-4c5f-8078-32de22603c74 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:15.434517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:15.436466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:15.436466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:15.436822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] PATH_INFO: `/v3/projects/92f16adc2357459aa0afb68db89031dd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:15.440886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f16adc2357459aa0afb68db89031dd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:15.442213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:15.599871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f16adc2357459aa0afb68db89031dd', 'request_id': 'req-e168a6a2-1f5b-40ff-8de1-bd74935dc735'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:15.616161 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f16adc2357459aa0afb68db89031dd', 'request_id': 'req-e168a6a2-1f5b-40ff-8de1-bd74935dc735'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:15.638376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f16adc2357459aa0afb68db89031dd', 'request_id': 'req-e168a6a2-1f5b-40ff-8de1-bd74935dc735'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:15.668763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] The token cache is being invalidate because project 92f16adc2357459aa0afb68db89031dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:15.668763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:15.670782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e168a6a2-1f5b-40ff-8de1-bd74935dc735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:15.678409 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2522] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:15 2022] DELETE /identity/v3/projects/92f16adc2357459aa0afb68db89031dd => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:15.680694 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:15.682758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:15.685389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:15.741962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:15.743841 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:15.831388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:15.866622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1282/2523] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:15.870385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:15.870907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:15.871616 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:15.922281 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:15.925594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:15.948945 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2524] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:15.949819 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:15.963015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45b0a0-63d0-404e-97e3-fd875498dece None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:15.969334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:15.970129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:15.970550 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:15.971470 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:15.972368 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:15.972756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:15.975874 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:15.998881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1283/2525] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:28:16.001030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.018696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd80d-4d5d-41cf-96e3-38c5137aa0d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.027744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.029895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.030357 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.031130 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.032418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.037139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:16.088078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2526] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:15 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:16.089587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.113492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069576bd-a1d6-4f0f-9b91-a048b2d3d8ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.244292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.244940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.245233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.245643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.246171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.249911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:16.287290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1284/2527] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:16 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:16.293143 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.305539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22219a4e-891d-4948-9daa-bbf5388634c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.310986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.311697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.312003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.312267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.313785 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.314687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:16.328667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2528] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:16.333204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.347074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b47645-dc81-4bce-83e4-8be0c774d56a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.354698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e04f4c-5d06-4476-b01d-4e7298d4f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.356823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e04f4c-5d06-4476-b01d-4e7298d4f856 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.357300 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e04f4c-5d06-4476-b01d-4e7298d4f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.357625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e04f4c-5d06-4476-b01d-4e7298d4f856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.358436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e04f4c-5d06-4476-b01d-4e7298d4f856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.360601 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e04f4c-5d06-4476-b01d-4e7298d4f856 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:16.375428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1285/2529] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:16.380278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.393301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af3704-c53a-4de0-9fb2-4c162bd4cf23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.401602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.402407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.402768 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.403119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] PATH_INFO: `/v3/projects/e5057b6e4d6744f195a35849de1cee6f/users/5f52be3192974e3fa636f17531432b3d/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.414917 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:16.425354 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5057b6e4d6744f195a35849de1cee6f, user_id=5f52be3192974e3fa636f17531432b3d, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.428932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:16.451976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2530] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:16 2022] PUT /identity/v3/projects/e5057b6e4d6744f195a35849de1cee6f/users/5f52be3192974e3fa636f17531432b3d/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:16.469342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.469824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.470267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.529986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] MFA Rules not processed for user `5f52be3192974e3fa636f17531432b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:16.531751 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:16.659006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1286/2531] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:16.663846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.765095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d17bd1-cb37-47b5-bded-d25ff09a8150 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.776120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.781088 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.781527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.781914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.783092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.785342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:16.824269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2532] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:16 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:16.831284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:16.854747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343d7c7-45d5-4a04-95c8-d4afc2338d26 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:16.960903 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:16.982283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:16.982795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:16.982962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:16.983173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:16.984191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:16.987206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:17.043567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1287/2533] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:16 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:17.045796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:17.069366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543bfa-c893-48c2-a05a-5ebd10b894f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:17.078192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:17.079293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:17.079836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:17.080311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:17.081603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:17.084755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:17.105854 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2534] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:17.112572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:17.139872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540d0e-02ea-40fd-bb61-67f8e22f81ce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:17.149320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:17.150993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:17.151247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:17.151827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] PATH_INFO: `/v3/projects/9dacc2bf82e045daa50ba2f92b8f391b/users/b3f306a911124b6fa9305eb1f5ecad8c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:17.184072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dacc2bf82e045daa50ba2f92b8f391b, user_id=b3f306a911124b6fa9305eb1f5ecad8c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:17.188636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:17.232761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1288/2535] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:17 2022] PUT /identity/v3/projects/9dacc2bf82e045daa50ba2f92b8f391b/users/b3f306a911124b6fa9305eb1f5ecad8c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:17.234038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:17.246432 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c267de-b30d-4c1c-bf95-8ae7a5c6d0bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:17.337628 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a3d24-d74c-4e70-b994-38e980f223ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:17.338515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a3d24-d74c-4e70-b994-38e980f223ca admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:17.338952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a3d24-d74c-4e70-b994-38e980f223ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:17.339386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a3d24-d74c-4e70-b994-38e980f223ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:17.340207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a3d24-d74c-4e70-b994-38e980f223ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:17.342795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a3d24-d74c-4e70-b994-38e980f223ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:17.355402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2536] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:17.360146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:17.377203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d899-e47c-4842-bbf0-28871ac4807e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:17.388292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:17.390926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:17.391486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:17.392002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] PATH_INFO: `/v3/projects/9dacc2bf82e045daa50ba2f92b8f391b/users/b3f306a911124b6fa9305eb1f5ecad8c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:17.395933 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dacc2bf82e045daa50ba2f92b8f391b, user_id=b3f306a911124b6fa9305eb1f5ecad8c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:17.399735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:17.415523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1289/2537] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:17 2022] PUT /identity/v3/projects/9dacc2bf82e045daa50ba2f92b8f391b/users/b3f306a911124b6fa9305eb1f5ecad8c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:17.428050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:17.432085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:17.432840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:17.491662 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] MFA Rules not processed for user `b3f306a911124b6fa9305eb1f5ecad8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:17.493516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:17.617980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:17.688988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2538] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:17.712867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:17.717508 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:17.863077 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64201-2121-46f2-a040-84e300c031a8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:17.868266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3b1c6-8745-4d7b-8e0d-ff6b30e3f3a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:17.868930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3b1c6-8745-4d7b-8e0d-ff6b30e3f3a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:17.869246 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3b1c6-8745-4d7b-8e0d-ff6b30e3f3a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:17.869526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3b1c6-8745-4d7b-8e0d-ff6b30e3f3a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:17.905031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3b1c6-8745-4d7b-8e0d-ff6b30e3f3a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:17.907010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3b1c6-8745-4d7b-8e0d-ff6b30e3f3a4 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:17.923116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1290/2539] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:18.052244 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:18.065763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a955e-2afa-4af2-92c8-3691dc24d879 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:18.069653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:18.070298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:18.070588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:18.070869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:18.150690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:18.152519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:18.182872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2540] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:19.192337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:19.193338 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:19.193787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:19.248375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] MFA Rules not processed for user `5b1d0da1157a46c283bd442ad14d31b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:19.251136 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:19.389926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1291/2541] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:19.395947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:19.397549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:19.508381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9e54-9aa8-4dbe-abcd-339d293193d0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:19.516656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:19.517605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:19.518083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:19.518525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:19.519564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:19.521756 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:19.566659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2542] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:19 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:19.566820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:19.605114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34eeb3-ce05-411f-b7e5-39fc0b11c941 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:19.712981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:19.788302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:19.789233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:19.789546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:19.790902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:19.791676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:19.796137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:19.845039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1292/2543] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:19 2022] POST /identity/v3/users => generated 292 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:19.853585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:19.879383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a66be-e34b-4b83-a5aa-22049e84a008 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:19.890428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:19.891602 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:19.892111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:19.892597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:19.893852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:19.897562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:19.923167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2544] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:19.928510 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:19.945410 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe177503-9565-4b5f-b5ab-6244d43572be admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:19.954049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:19.954769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:19.955063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:19.955492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] PATH_INFO: `/v3/projects/ab146e39de994e9db7b6e4e9b773d4c7/users/8921309a07e24aca97786b9faead7d98/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:19.977826 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab146e39de994e9db7b6e4e9b773d4c7, user_id=8921309a07e24aca97786b9faead7d98, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:19.983143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:20.010474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1293/2545] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:19 2022] PUT /identity/v3/projects/ab146e39de994e9db7b6e4e9b773d4c7/users/8921309a07e24aca97786b9faead7d98/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:20.016414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:20.028979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9cf17-a36b-4052-a9f6-2df02e7501ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:20.100759 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:20.114765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6b965-ecd9-4581-b5d7-25ea4468ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:20.116708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6b965-ecd9-4581-b5d7-25ea4468ff92 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:20.118054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6b965-ecd9-4581-b5d7-25ea4468ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:20.118563 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6b965-ecd9-4581-b5d7-25ea4468ff92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:20.121043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6b965-ecd9-4581-b5d7-25ea4468ff92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:20.126007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6b965-ecd9-4581-b5d7-25ea4468ff92 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:20.154465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2546] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:20.161455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:20.188237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a740c-0305-4e60-8f42-197915627549 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:20.210756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:20.212202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:20.212975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:20.213740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] PATH_INFO: `/v3/projects/ab146e39de994e9db7b6e4e9b773d4c7/users/8921309a07e24aca97786b9faead7d98/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:20.221810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab146e39de994e9db7b6e4e9b773d4c7, user_id=8921309a07e24aca97786b9faead7d98, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:20.227274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:20.255610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1294/2547] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:20 2022] PUT /identity/v3/projects/ab146e39de994e9db7b6e4e9b773d4c7/users/8921309a07e24aca97786b9faead7d98/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:25.866633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:25.867253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:25.867253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:25.909991 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] MFA Rules not processed for user `8921309a07e24aca97786b9faead7d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:25.913375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:25.985630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:25.989378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:26.073689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dceece-95a2-4718-8049-b589de98a852 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:26.079156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:26.080274 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:26.080795 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:26.082265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:26.082496 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:26.086454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:26.092524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:26.094905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:26.205199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:26.206948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:26.236499 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1295/2549] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:26.288483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf52c1-5092-4cfd-a626-647dd9decb48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:26.296820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58841b6c-b811-458c-81d5-b4869c2e9b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:26.297739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58841b6c-b811-458c-81d5-b4869c2e9b69 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:26.298098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58841b6c-b811-458c-81d5-b4869c2e9b69 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:26.298514 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58841b6c-b811-458c-81d5-b4869c2e9b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:26.423221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58841b6c-b811-458c-81d5-b4869c2e9b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:26.425229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58841b6c-b811-458c-81d5-b4869c2e9b69 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:26.438342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:27.628642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:27.641227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bb445-82d8-4265-83d1-d8b5142c6f7a service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:27.655375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:27.656502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:27.657022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:27.657558 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] PATH_INFO: `/v3/users/5f52be3192974e3fa636f17531432b3d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:27.661125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f52be3192974e3fa636f17531432b3d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:27.663270 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:27.736566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:27.767762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f52be3192974e3fa636f17531432b3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:27.781122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f52be3192974e3fa636f17531432b3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:27.784185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f52be3192974e3fa636f17531432b3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:27.786847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f52be3192974e3fa636f17531432b3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:27.806702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f52be3192974e3fa636f17531432b3d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:27.827162 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1296/2551] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:27 2022] DELETE /identity/v3/users/5f52be3192974e3fa636f17531432b3d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:27.830966 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc6e8-cc55-440e-9842-9917a39b49be None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:27.831496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc6e8-cc55-440e-9842-9917a39b49be None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:27.831846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc6e8-cc55-440e-9842-9917a39b49be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:27.856533 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0f1bc6e8-cc55-440e-9842-9917a39b49be None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:27.858639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f1bc6e8-cc55-440e-9842-9917a39b49be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:27.870109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:27.946070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2552] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:27.964471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:27.968188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:28.149481 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2047ed-8639-4753-b168-3c0125eb4667 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:28.161111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:28.162225 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:28.162798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:28.163299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:28.185633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:28.190406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:28.211625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1297/2553] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:28:28.247328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:28.247979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:28.248539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:28.272855 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] MFA Rules not processed for user `5b1d0da1157a46c283bd442ad14d31b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:28.275048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:28.366828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2554] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:28.740502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:28.743497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:28.769807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822150f4-b09d-49d6-815c-26b832d6e3d2 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:28.778766 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:28.779101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:28.779596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:28.779956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] PATH_INFO: `/v3/projects/e5057b6e4d6744f195a35849de1cee6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:28.781361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5057b6e4d6744f195a35849de1cee6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:28.783618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:28.858808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5057b6e4d6744f195a35849de1cee6f', 'request_id': 'req-04c7baf3-b753-48c1-a474-004e1f83bd91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:28.870960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5057b6e4d6744f195a35849de1cee6f', 'request_id': 'req-04c7baf3-b753-48c1-a474-004e1f83bd91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:28.881450 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5057b6e4d6744f195a35849de1cee6f', 'request_id': 'req-04c7baf3-b753-48c1-a474-004e1f83bd91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:28.900222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] The token cache is being invalidate because project e5057b6e4d6744f195a35849de1cee6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:28.900695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:28.904459 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:28.908244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1298/2555] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:28 2022] DELETE /identity/v3/projects/e5057b6e4d6744f195a35849de1cee6f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:28.913539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:28.936269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980dbc-3018-4af3-8d48-5b7fd9a80747 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:29.066510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:29.067555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:29.068052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:29.068632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] PATH_INFO: `/v3/users/b3f306a911124b6fa9305eb1f5ecad8c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:29.070165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f306a911124b6fa9305eb1f5ecad8c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:29.072780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:29.162293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f306a911124b6fa9305eb1f5ecad8c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:29.163766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f306a911124b6fa9305eb1f5ecad8c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:29.178668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f306a911124b6fa9305eb1f5ecad8c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:29.179327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f306a911124b6fa9305eb1f5ecad8c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:29.185201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:29.201724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-278442a6-91a8-4585-bf7b-cae7553dd215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f306a911124b6fa9305eb1f5ecad8c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:29.227555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2556] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:28 2022] DELETE /identity/v3/users/b3f306a911124b6fa9305eb1f5ecad8c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:29.931998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:29.933761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:29.958916 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7baf3-b753-48c1-a474-004e1f83bd91 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:30.032399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:30.093301 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:30.094522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:30.095205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:30.095634 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] PATH_INFO: `/v3/projects/9dacc2bf82e045daa50ba2f92b8f391b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:30.098135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dacc2bf82e045daa50ba2f92b8f391b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:30.100367 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:30.316494 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dacc2bf82e045daa50ba2f92b8f391b', 'request_id': 'req-f7f53a68-5070-490a-a408-00712532afdf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:30.352561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dacc2bf82e045daa50ba2f92b8f391b', 'request_id': 'req-f7f53a68-5070-490a-a408-00712532afdf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:30.380538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dacc2bf82e045daa50ba2f92b8f391b', 'request_id': 'req-f7f53a68-5070-490a-a408-00712532afdf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:30.429544 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] The token cache is being invalidate because project 9dacc2bf82e045daa50ba2f92b8f391b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:30.430985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:30.432486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7f53a68-5070-490a-a408-00712532afdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:30.437492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1299/2557] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:29 2022] DELETE /identity/v3/projects/9dacc2bf82e045daa50ba2f92b8f391b => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:30.445874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:30.446381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:30.446496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:30.491398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:30.493831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:30.651741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2558] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:30.660583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:30.661217 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:30.661719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:30.730944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:30.733058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:30.744103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1300/2559] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:30.750690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:30.785513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20586-974f-4f8a-a4f0-91e3b4795c92 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:30.804967 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:30.805912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:30.806890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:30.807386 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:30.808381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:30.809531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:30.812224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:30.840623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2560] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:28:30.841865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:30.853684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8d1bd-4180-4b34-a1a3-fdf23d5a79c7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:30.885197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:30.888256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:30.888256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:30.888256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:30.888256 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:30.901412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.062823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.067069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1301/2561] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:30 2022] POST /identity/v3/projects => generated 369 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:31.096254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85631801-3cd8-4864-9cbb-ad38119d3bc3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.255335 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.256033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.256278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.256529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.257230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.259480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.293957 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2562] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:31 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:31.302084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.308101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.310090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:31.326882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2860c-2bda-4730-b5f5-b63c4984aef8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.334854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.336576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.337798 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.338678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.345769 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.348165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.366103 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1302/2563] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:31.367669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.369894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:31.409320 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14c83-b12d-485a-9dce-a23740066664 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.413212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:31.415573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.416358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.416660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.416981 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] PATH_INFO: `/v3/users/3c60b3a983f14f50b1b762a3954a1059` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.418965 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c60b3a983f14f50b1b762a3954a1059)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.421482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.432948 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a1cb-65b0-40d8-a3c3-1cf37ed57b2a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.439875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.441324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.441827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.442247 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.445105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:31.462253 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.464164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.479970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2564] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:31.481391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.497259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8972-390b-4341-be41-8172595fa7fd service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.506579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.507605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.507893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.508180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.508882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.510517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.525721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2565] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:31.530640 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.547480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e65d5-6ee3-4947-a571-676df3a99e75 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.554741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6aee72-e54f-42d2-a4d9-f7fea0cd1b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.555475 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6aee72-e54f-42d2-a4d9-f7fea0cd1b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.555980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6aee72-e54f-42d2-a4d9-f7fea0cd1b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.556268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6aee72-e54f-42d2-a4d9-f7fea0cd1b7c admin admin] PATH_INFO: `/v3/projects/a3c16e58753c41149a3f8f2ef4ce1b0d/users/b249dd9d75504505ad67fa100fc3cdf7/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.557745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c60b3a983f14f50b1b762a3954a1059'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:31.574746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c60b3a983f14f50b1b762a3954a1059'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:31.576619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c60b3a983f14f50b1b762a3954a1059'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:31.578117 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c60b3a983f14f50b1b762a3954a1059'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:31.581080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6aee72-e54f-42d2-a4d9-f7fea0cd1b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c16e58753c41149a3f8f2ef4ce1b0d, user_id=b249dd9d75504505ad67fa100fc3cdf7, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.586796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6aee72-e54f-42d2-a4d9-f7fea0cd1b7c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.604961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c60b3a983f14f50b1b762a3954a1059'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:31.614285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:31.633366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2566] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:31 2022] PUT /identity/v3/projects/a3c16e58753c41149a3f8f2ef4ce1b0d/users/b249dd9d75504505ad67fa100fc3cdf7/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:31.636411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.636773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.637165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.651764 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1303/2567] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:31 2022] DELETE /identity/v3/users/3c60b3a983f14f50b1b762a3954a1059 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:31.675490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] MFA Rules not processed for user `b249dd9d75504505ad67fa100fc3cdf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:31.676894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:31.791535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2568] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:31.806214 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.807777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:31.916035 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6c55a-1b30-4c7c-b66a-7f1b58f9c4a0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:31.923241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4676aa-03fd-4eb7-b340-457c961a7a66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c858ed2d88a43afaf34e83155629c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:31.924468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4676aa-03fd-4eb7-b340-457c961a7a66 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:31.925001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4676aa-03fd-4eb7-b340-457c961a7a66 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:31.925528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4676aa-03fd-4eb7-b340-457c961a7a66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:31.942419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4676aa-03fd-4eb7-b340-457c961a7a66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:31.944115 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4676aa-03fd-4eb7-b340-457c961a7a66 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:31.956028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1304/2569] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:31.956460 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:31.957877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:32.062586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84208d2a-6705-4f91-9a70-53a8a6a212d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.068096 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.071709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.071982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.072270 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] PATH_INFO: `/v3/projects/9d8baaf2e23546ee8ed1de278617c7d2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.073714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8baaf2e23546ee8ed1de278617c7d2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.076712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.154841 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:32.163577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8baaf2e23546ee8ed1de278617c7d2', 'request_id': 'req-322a71f3-9d57-4228-a50c-b9c24ad0ee99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:32.175186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8baaf2e23546ee8ed1de278617c7d2', 'request_id': 'req-322a71f3-9d57-4228-a50c-b9c24ad0ee99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:32.185627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8baaf2e23546ee8ed1de278617c7d2', 'request_id': 'req-322a71f3-9d57-4228-a50c-b9c24ad0ee99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:32.210675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] The token cache is being invalidate because project 9d8baaf2e23546ee8ed1de278617c7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:32.210982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:32.211722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-322a71f3-9d57-4228-a50c-b9c24ad0ee99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:32.214013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2570] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:31 2022] DELETE /identity/v3/projects/9d8baaf2e23546ee8ed1de278617c7d2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:32.222005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.222577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.222918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.244996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:32.247660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:32.325400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1305/2571] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:32.334419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.334943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.335442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.359589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:32.360879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:32.373778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2572] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:32.379760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:32.400790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aeeef-c75a-4d10-ae2a-6b391dd7fb3a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.408076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.408741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.409030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.409313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.410221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:32.410401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.412106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.430913 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1306/2573] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:28:32.436540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:32.450579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a74ff8-7c08-47cf-bb09-8e21d7b60322 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.464807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.466446 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.466738 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.467596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.468534 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.472560 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.531912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2574] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:32 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:32.544700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:32.569436 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5ead5-2c37-4101-8a7f-5fb9703b1b20 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.681897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.682554 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.682842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.683120 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.685157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.687133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.718710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1307/2575] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:32 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:32.722959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:32.735685 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d165533-89e2-4541-858c-86cbb40924ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.748093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.748749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.749036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.749319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.750037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.751712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.766899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2576] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:32.771133 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:32.783657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bce433-8c53-4044-8b82-d2007951a240 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.795732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e12e4-d3c6-462f-bd85-ed8cf54dc462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.796527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e12e4-d3c6-462f-bd85-ed8cf54dc462 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.796823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e12e4-d3c6-462f-bd85-ed8cf54dc462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.797108 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e12e4-d3c6-462f-bd85-ed8cf54dc462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.797860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e12e4-d3c6-462f-bd85-ed8cf54dc462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.800247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e12e4-d3c6-462f-bd85-ed8cf54dc462 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.815711 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1308/2577] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:32.820226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:32.832860 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ca602-f4c9-43d5-a6fa-61e20d46d87d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:32.838961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:32.839796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.840178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.840692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] PATH_INFO: `/v3/projects/c74e3cde3825470f96c13aae2b719891/users/eeaf5c65288a49d799d8b299b6a46a36/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.858930 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74e3cde3825470f96c13aae2b719891, user_id=eeaf5c65288a49d799d8b299b6a46a36, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:32.862277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:32.885965 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2578] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:32 2022] PUT /identity/v3/projects/c74e3cde3825470f96c13aae2b719891/users/eeaf5c65288a49d799d8b299b6a46a36/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:32.896250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:32.896547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:32.896822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:32.904997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:32.933594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] MFA Rules not processed for user `eeaf5c65288a49d799d8b299b6a46a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:32.935765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:33.041773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1309/2579] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:33.046803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.130180 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a255d9-e5ce-43a2-a5b9-72f95e322add admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.135843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.136526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.137370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.137683 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.138429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.140631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.178943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2580] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:33 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:33.186538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.198298 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5d146-5643-4b47-b1e7-ed8280481ddf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.269358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.270072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.270269 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.270552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.271228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.273228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.297171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1310/2581] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:33 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:33.302094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.315004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a86a2-da73-4ef0-931e-51fd54cc9fa3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.320072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.320720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.321004 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.321284 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.322009 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.323740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.336597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2582] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:33.341432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.356522 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae240a-8db3-4a49-831a-725d5f43d553 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.364429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.365642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.366229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.366639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] PATH_INFO: `/v3/projects/895e2712257c496a90b285bd5f3b3121/users/e7f5d74371964734ac71778b9015ad2e/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.390891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=895e2712257c496a90b285bd5f3b3121, user_id=e7f5d74371964734ac71778b9015ad2e, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.393630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.408537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1311/2583] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:33 2022] PUT /identity/v3/projects/895e2712257c496a90b285bd5f3b3121/users/e7f5d74371964734ac71778b9015ad2e/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:33.413136 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.434094 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44427b2-ba8f-4d8c-96aa-f7aaef95df2d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.478527 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:33.502728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871a21a-2619-4c93-a306-973c386d24b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.503370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871a21a-2619-4c93-a306-973c386d24b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.503671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871a21a-2619-4c93-a306-973c386d24b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.503950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871a21a-2619-4c93-a306-973c386d24b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.504646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871a21a-2619-4c93-a306-973c386d24b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.506294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871a21a-2619-4c93-a306-973c386d24b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.516766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2584] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:33.521173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.532622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0598b3-5df0-4ecc-b92e-4a48405dbff5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.537398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.538085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.538375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.538659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] PATH_INFO: `/v3/projects/895e2712257c496a90b285bd5f3b3121/users/e7f5d74371964734ac71778b9015ad2e/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.540677 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=895e2712257c496a90b285bd5f3b3121, user_id=e7f5d74371964734ac71778b9015ad2e, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.543977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.555404 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1312/2585] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:33 2022] PUT /identity/v3/projects/895e2712257c496a90b285bd5f3b3121/users/e7f5d74371964734ac71778b9015ad2e/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:33.569719 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.570129 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.570571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.607402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] MFA Rules not processed for user `e7f5d74371964734ac71778b9015ad2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:33.611414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:33.706788 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2586] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:33.721339 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.723111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:33.776703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:33.789535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030832-8734-46db-9ea7-b851e0f8e88f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:33.794151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab935948-7b14-434b-9c99-bade0d69f73d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:33.794782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935948-7b14-434b-9c99-bade0d69f73d service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:33.795066 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935948-7b14-434b-9c99-bade0d69f73d service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:33.795400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935948-7b14-434b-9c99-bade0d69f73d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:33.806181 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab935948-7b14-434b-9c99-bade0d69f73d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:33.807937 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab935948-7b14-434b-9c99-bade0d69f73d service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:33.819271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1313/2587] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:33.907390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:33.996824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e64787-1ddf-43b2-91c8-3cb5bdc59f4c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:34.004963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:34.005652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:34.005934 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:34.006221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] PATH_INFO: `/v3/users/eeaf5c65288a49d799d8b299b6a46a36` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:34.007488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeaf5c65288a49d799d8b299b6a46a36)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:34.009507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:34.070532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeaf5c65288a49d799d8b299b6a46a36'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.071257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeaf5c65288a49d799d8b299b6a46a36'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.085391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeaf5c65288a49d799d8b299b6a46a36'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.086153 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeaf5c65288a49d799d8b299b6a46a36'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.091802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:34.101749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeaf5c65288a49d799d8b299b6a46a36'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.116301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2588] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:33 2022] DELETE /identity/v3/users/eeaf5c65288a49d799d8b299b6a46a36 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:34.122647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:34.122647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:34.123105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:34.146188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:34.147573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:34.227710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1314/2589] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:34.241601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:34.241931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:34.306037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb915c-76be-49be-b655-2fc424ad03be admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:34.310598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:34.311227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:34.311545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:34.311813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:34.321880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:34.323365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:34.333798 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2590] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:34.721881 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:34.728261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:34.745188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc51d62-2b5a-41d2-832d-39218b1a5892 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:34.753952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:34.755201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:34.755918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:34.756435 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] PATH_INFO: `/v3/projects/c74e3cde3825470f96c13aae2b719891` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:34.758778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74e3cde3825470f96c13aae2b719891)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:34.763393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:34.839112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74e3cde3825470f96c13aae2b719891', 'request_id': 'req-83fced83-cb9c-49dc-a205-9b4c5a6bc251'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.849010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74e3cde3825470f96c13aae2b719891', 'request_id': 'req-83fced83-cb9c-49dc-a205-9b4c5a6bc251'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.856097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74e3cde3825470f96c13aae2b719891', 'request_id': 'req-83fced83-cb9c-49dc-a205-9b4c5a6bc251'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.868507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] The token cache is being invalidate because project c74e3cde3825470f96c13aae2b719891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:34.868804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.869594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:34.872268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1315/2591] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:34 2022] DELETE /identity/v3/projects/c74e3cde3825470f96c13aae2b719891 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:34.877366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:34.895055 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14969e6-d9bf-4086-847f-e6a1e60e57cc service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:34.964520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:34.965710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:34.965996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:34.966277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] PATH_INFO: `/v3/users/e7f5d74371964734ac71778b9015ad2e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:34.967567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f5d74371964734ac71778b9015ad2e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:34.969559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:35.034879 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f5d74371964734ac71778b9015ad2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.035559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f5d74371964734ac71778b9015ad2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.043334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f5d74371964734ac71778b9015ad2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.044440 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f5d74371964734ac71778b9015ad2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.065512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e5d16fb-83b1-4407-b58f-9fa5be8de5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f5d74371964734ac71778b9015ad2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.080570 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2592] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:34 2022] DELETE /identity/v3/users/e7f5d74371964734ac71778b9015ad2e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:35.440958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:35.442669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:35.459939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fced83-cb9c-49dc-a205-9b4c5a6bc251 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:35.527754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:35.528425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:35.528713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:35.528997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] PATH_INFO: `/v3/projects/895e2712257c496a90b285bd5f3b3121` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:35.530210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=895e2712257c496a90b285bd5f3b3121)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:35.532239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:35.545413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:35.597877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895e2712257c496a90b285bd5f3b3121', 'request_id': 'req-02e6c70b-0369-40f5-bb54-dcd92da41aaf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.607649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895e2712257c496a90b285bd5f3b3121', 'request_id': 'req-02e6c70b-0369-40f5-bb54-dcd92da41aaf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.618952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895e2712257c496a90b285bd5f3b3121', 'request_id': 'req-02e6c70b-0369-40f5-bb54-dcd92da41aaf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.633849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] The token cache is being invalidate because project 895e2712257c496a90b285bd5f3b3121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:35.634147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.634854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02e6c70b-0369-40f5-bb54-dcd92da41aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:35.637039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1316/2593] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:35 2022] DELETE /identity/v3/projects/895e2712257c496a90b285bd5f3b3121 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:35.647897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:35.648655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:35.649127 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:35.664271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:35.682665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:35.683995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:35.786196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2594] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:35.800457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:35.800457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:35.800457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:35.821547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:35.822760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:35.840507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1317/2595] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:35.842728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:35.859520 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f8dc-4930-4434-b013-bc35690c02c1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:35.864480 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:35.865186 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:35.865542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:35.865891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:35.866663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:35.867041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:35.868804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:35.886864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2596] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:28:35.893051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:35.905054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173e0db-c9c2-41d1-8dc4-8b2b49b72cfe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:35.910414 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:35.911161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:35.912039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:35.912039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:35.912374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:35.914280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:35.947265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1318/2597] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:35 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:35.949555 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:35.960709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a40e1b-4770-4582-90dc-47d7e2ee4d08 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.032463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.033098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.033385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.033659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.034377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.036383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.061305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2598] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:35 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:36.062700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.075277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a200-aab7-4635-9e31-09e9345ad840 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.080171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.080804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.081086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.081364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.082058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.083722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.097309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1319/2599] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:36.101086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.111892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b97486-ab6e-4789-8f16-345a0320d325 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.118046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a87d8c-9825-4692-b819-5a974aaf7a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.118936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a87d8c-9825-4692-b819-5a974aaf7a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.119202 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a87d8c-9825-4692-b819-5a974aaf7a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.119296 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a87d8c-9825-4692-b819-5a974aaf7a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.119997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a87d8c-9825-4692-b819-5a974aaf7a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.122952 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a87d8c-9825-4692-b819-5a974aaf7a76 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.139510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2600] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:36.144573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.157678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc039aaa-68ce-4b65-bd9e-cd65f22fa6c3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.162525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.163195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.163524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.163836 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] PATH_INFO: `/v3/projects/4bd5a61429914dbc97ec7744d1c00ac4/users/3f59b3919df641349dd9e5170424c6ad/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.183627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd5a61429914dbc97ec7744d1c00ac4, user_id=3f59b3919df641349dd9e5170424c6ad, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.186823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.205914 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1320/2601] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:36 2022] PUT /identity/v3/projects/4bd5a61429914dbc97ec7744d1c00ac4/users/3f59b3919df641349dd9e5170424c6ad/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:36.218604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.218904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.219192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.257207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] MFA Rules not processed for user `3f59b3919df641349dd9e5170424c6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:36.258467 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:36.341642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:36.350140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2602] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:36.354971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.425761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2069c-c33a-45c5-9054-8ea39937f1b2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.431403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.432145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.432438 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.432722 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.433531 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.435525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.474974 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1321/2603] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:36 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:36.476490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.488398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d15ee-873f-4589-a6bc-4bcb61565ac8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.554356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.554992 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.555280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.555610 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.556291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.558938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.589211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2604] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:36 2022] POST /identity/v3/users => generated 306 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:36.593742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.607754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbeb5-c6d7-4f23-b997-164abeb3d26f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.613447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.614139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.614423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.614703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.615487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.617211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.630434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1322/2605] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:36.635090 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.648859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246b73c-05bf-4594-9557-02e2e4b35ea5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.657298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.658030 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.658303 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.658586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] PATH_INFO: `/v3/projects/788f27b4b8574600a8c77577d27645dc/users/59ffd75aacc041f8bc82105cd3f73ede/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.677146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788f27b4b8574600a8c77577d27645dc, user_id=59ffd75aacc041f8bc82105cd3f73ede, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.680002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.698083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2606] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:36 2022] PUT /identity/v3/projects/788f27b4b8574600a8c77577d27645dc/users/59ffd75aacc041f8bc82105cd3f73ede/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:36.702749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.713753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5472bf-3e3d-4e60-aab9-137cc1f07f4d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.723460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:36.804202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edfda2-31a2-4999-8470-21bb6f46d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.805341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edfda2-31a2-4999-8470-21bb6f46d4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.805643 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edfda2-31a2-4999-8470-21bb6f46d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.806026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edfda2-31a2-4999-8470-21bb6f46d4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.806849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edfda2-31a2-4999-8470-21bb6f46d4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.808661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edfda2-31a2-4999-8470-21bb6f46d4ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.820249 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1323/2607] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:36.826268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:36.838269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9c0c4-624b-4bdd-af78-82bf0e959af6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:36.844667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:36.845368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.845655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.845940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] PATH_INFO: `/v3/projects/788f27b4b8574600a8c77577d27645dc/users/59ffd75aacc041f8bc82105cd3f73ede/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.848389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788f27b4b8574600a8c77577d27645dc, user_id=59ffd75aacc041f8bc82105cd3f73ede, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:36.851981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:36.882815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2608] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:36 2022] PUT /identity/v3/projects/788f27b4b8574600a8c77577d27645dc/users/59ffd75aacc041f8bc82105cd3f73ede/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:36.896658 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:36.897148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:36.897604 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:36.942310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] MFA Rules not processed for user `59ffd75aacc041f8bc82105cd3f73ede`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:36.944086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:37.026907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1324/2609] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:36 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:37.038144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:37.040379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:37.115902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ba869-fe44-4b2b-8ea9-52d35eb3dd37 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:37.123897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:37.125003 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:37.125515 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:37.126049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:37.140066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:37.141594 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:37.153036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2610] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:37.244138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:37.255109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f6c8-18ef-43be-839f-07fb5fe12e85 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:37.259046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33abad-cbd2-4baa-b71d-b69e7bb3375e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:37.259713 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33abad-cbd2-4baa-b71d-b69e7bb3375e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:37.260003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33abad-cbd2-4baa-b71d-b69e7bb3375e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:37.260285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33abad-cbd2-4baa-b71d-b69e7bb3375e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:37.324131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33abad-cbd2-4baa-b71d-b69e7bb3375e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:37.326137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33abad-cbd2-4baa-b71d-b69e7bb3375e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:37.338351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1325/2611] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:41.586741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:41.640916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:41.654873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c4d5-7577-409b-b394-9e9ed3288fa0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:41.659723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:41.660442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:41.660732 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:41.661080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] PATH_INFO: `/v3/users/3f59b3919df641349dd9e5170424c6ad` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:41.662287 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f59b3919df641349dd9e5170424c6ad)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:41.664411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:41.717189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f59b3919df641349dd9e5170424c6ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:41.717884 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f59b3919df641349dd9e5170424c6ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:41.724851 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f59b3919df641349dd9e5170424c6ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:41.725492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f59b3919df641349dd9e5170424c6ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:41.739704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f59b3919df641349dd9e5170424c6ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:41.756114 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2612] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:41 2022] DELETE /identity/v3/users/3f59b3919df641349dd9e5170424c6ad => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:41.760633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:41.761325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:41.761325 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:41.788944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:41.788944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:41.815380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:41.852283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1326/2613] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:41.864115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:41.865636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:41.930174 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b24aec-714c-44a3-bb41-113300103f0c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:41.934452 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:41.935085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:41.935423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:41.935717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:41.945858 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:41.947606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:41.958913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2614] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:42.276082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:42.277724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:42.289867 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a301d2-4583-44fc-a5dd-0ab393f469a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:42.294877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:42.295556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:42.295840 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:42.296116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] PATH_INFO: `/v3/projects/4bd5a61429914dbc97ec7744d1c00ac4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:42.297394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd5a61429914dbc97ec7744d1c00ac4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:42.299415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:42.348860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd5a61429914dbc97ec7744d1c00ac4', 'request_id': 'req-38f2b905-f4bd-420e-a594-8d0aa744f911'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.365689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd5a61429914dbc97ec7744d1c00ac4', 'request_id': 'req-38f2b905-f4bd-420e-a594-8d0aa744f911'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.373063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd5a61429914dbc97ec7744d1c00ac4', 'request_id': 'req-38f2b905-f4bd-420e-a594-8d0aa744f911'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.387244 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] The token cache is being invalidate because project 4bd5a61429914dbc97ec7744d1c00ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:42.387707 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.388271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.389854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1327/2615] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:42 2022] DELETE /identity/v3/projects/4bd5a61429914dbc97ec7744d1c00ac4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:42.394370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:42.405390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db8868-a9d4-4f79-bf25-a893bb8dcd83 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:42.489289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:42.490219 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:42.490510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:42.490792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] PATH_INFO: `/v3/users/59ffd75aacc041f8bc82105cd3f73ede` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:42.492036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ffd75aacc041f8bc82105cd3f73ede)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:42.494000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:42.512531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:42.548239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ffd75aacc041f8bc82105cd3f73ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.548946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ffd75aacc041f8bc82105cd3f73ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.556544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ffd75aacc041f8bc82105cd3f73ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.557471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ffd75aacc041f8bc82105cd3f73ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.571232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc72ee6a-a75f-4ce0-a8c3-506d608b089e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ffd75aacc041f8bc82105cd3f73ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:42.587301 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2616] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:42 2022] DELETE /identity/v3/users/59ffd75aacc041f8bc82105cd3f73ede => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:42.920872 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:42.923142 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:42.936296 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b905-f4bd-420e-a594-8d0aa744f911 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.012541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.013206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.013497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.013784 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] PATH_INFO: `/v3/projects/788f27b4b8574600a8c77577d27645dc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.014958 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] RBAC: Authorizing `identity:delete_project(project_id=788f27b4b8574600a8c77577d27645dc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.016929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.070110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788f27b4b8574600a8c77577d27645dc', 'request_id': 'req-20e6f984-674a-483b-a2d8-fe9a1acb377a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:43.079801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788f27b4b8574600a8c77577d27645dc', 'request_id': 'req-20e6f984-674a-483b-a2d8-fe9a1acb377a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:43.081107 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:43.086991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788f27b4b8574600a8c77577d27645dc', 'request_id': 'req-20e6f984-674a-483b-a2d8-fe9a1acb377a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:43.100493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] The token cache is being invalidate because project 788f27b4b8574600a8c77577d27645dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:43.100783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:43.101502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-20e6f984-674a-483b-a2d8-fe9a1acb377a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:43.102989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1328/2617] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:42 2022] DELETE /identity/v3/projects/788f27b4b8574600a8c77577d27645dc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:43.109614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.110058 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.110483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.140041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:43.141626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:43.237962 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2618] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:43.246567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.246859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.247143 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.269803 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:43.271034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:43.287772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1329/2619] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:43.294418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.305716 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d8b7-551a-4ea3-8921-f2434c480bef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.312898 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.317218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.317218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.317218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.317218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:43.317218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.318856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.334275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2620] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:28:43.338532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.351385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7adc0-157d-416a-9b96-38ce58b86210 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.354968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.355659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.356364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.356653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.357348 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.359307 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.392010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1330/2621] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:43.393190 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.406201 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10d197-55a7-436f-9151-0c00ce028f0c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.478540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.479176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.479516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.479802 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.480569 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.482430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.514782 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2622] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:43.518996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.531299 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d733d-f7bd-49b4-80a4-a2b50da4cf81 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.536059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.536686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.536969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.537278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.537977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.539630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.550778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1331/2623] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:43.555038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.566282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c866bc1-ba91-41a2-9786-a41b1d9afea4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.571388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a565cc9-dc62-4d07-b316-053b85be80b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.572035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a565cc9-dc62-4d07-b316-053b85be80b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.572323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a565cc9-dc62-4d07-b316-053b85be80b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.572604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a565cc9-dc62-4d07-b316-053b85be80b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.573871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a565cc9-dc62-4d07-b316-053b85be80b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.575922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a565cc9-dc62-4d07-b316-053b85be80b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.581853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:43.588609 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2624] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:43.594079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.605618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671baa7-d35b-4a80-89cc-b6aceff9f120 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.613408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.614109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.614398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.614783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] PATH_INFO: `/v3/projects/1b2108031095477199883c48cfc39522/users/d4ea80e18dc1429f953ec638694778b9/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.634704 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2108031095477199883c48cfc39522, user_id=d4ea80e18dc1429f953ec638694778b9, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.638574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.654967 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1332/2625] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:43 2022] PUT /identity/v3/projects/1b2108031095477199883c48cfc39522/users/d4ea80e18dc1429f953ec638694778b9/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:43.667437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.667975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.668256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.703423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] MFA Rules not processed for user `d4ea80e18dc1429f953ec638694778b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:43.704715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:43.776189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2626] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:43.781374 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.848959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2622db-3aa6-41e7-a508-a580130ad6f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.853753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.854383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.854768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.855052 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.855845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.857823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.887635 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1333/2627] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:43.891714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:43.903069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ce818-b7ab-4efd-9b86-7d3c351cce06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:43.967500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:43.968140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:43.968428 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:43.968705 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:43.969398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:43.971390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:43.995080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2628] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:43 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:44.000526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.011177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47453586-6735-4fb0-93c1-a841b2cebc60 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.016012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.016640 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.016923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.017209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.017912 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.019560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.031366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1334/2629] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:44.035623 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.046524 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4a92d-10eb-4840-a8e8-4258d6993d13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.051922 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.052612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.053039 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.053333 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] PATH_INFO: `/v3/projects/3de3473157a54442822ac0c6ed2c6fc6/users/a32bd01a35bb469a93a2bcf80ffade4b/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.071106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de3473157a54442822ac0c6ed2c6fc6, user_id=a32bd01a35bb469a93a2bcf80ffade4b, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.074360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.091294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2630] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:44 2022] PUT /identity/v3/projects/3de3473157a54442822ac0c6ed2c6fc6/users/a32bd01a35bb469a93a2bcf80ffade4b/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:44.097169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.109371 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a17f0-4c46-4947-89a1-302edc15faf9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.178466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:44.185622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829e9eb-5fe4-418a-a735-5e6158d0291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.186562 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829e9eb-5fe4-418a-a735-5e6158d0291f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.186929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829e9eb-5fe4-418a-a735-5e6158d0291f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.187663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829e9eb-5fe4-418a-a735-5e6158d0291f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.188818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829e9eb-5fe4-418a-a735-5e6158d0291f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.191135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829e9eb-5fe4-418a-a735-5e6158d0291f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.203333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1335/2631] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:44.207305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.219291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8fdb1-1905-45c4-84e1-5e13739ea5aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.224105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.224805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.225092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.225376 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] PATH_INFO: `/v3/projects/3de3473157a54442822ac0c6ed2c6fc6/users/a32bd01a35bb469a93a2bcf80ffade4b/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.227435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de3473157a54442822ac0c6ed2c6fc6, user_id=a32bd01a35bb469a93a2bcf80ffade4b, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.230163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.240455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2632] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:44 2022] PUT /identity/v3/projects/3de3473157a54442822ac0c6ed2c6fc6/users/a32bd01a35bb469a93a2bcf80ffade4b/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:44.251144 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.251487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.251775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.281820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] MFA Rules not processed for user `a32bd01a35bb469a93a2bcf80ffade4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:44.283011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:44.356139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1336/2633] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:44.366936 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.368835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:44.421154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:44.438400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da3af5-2f7e-4277-86fb-2ae453e25b13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.442911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2ca9c-a384-46cf-8f2d-0b083ee7c1af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.443781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2ca9c-a384-46cf-8f2d-0b083ee7c1af service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.444067 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2ca9c-a384-46cf-8f2d-0b083ee7c1af service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.444358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2ca9c-a384-46cf-8f2d-0b083ee7c1af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.455916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2ca9c-a384-46cf-8f2d-0b083ee7c1af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.457380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2ca9c-a384-46cf-8f2d-0b083ee7c1af service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.469209 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2634] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:44.607996 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.673866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7a6b6-0cc6-452a-9015-f5cdce229da9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.678557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.679208 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.679527 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.679809 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] PATH_INFO: `/v3/users/d4ea80e18dc1429f953ec638694778b9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.680976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ea80e18dc1429f953ec638694778b9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.683526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.731821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ea80e18dc1429f953ec638694778b9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:44.739361 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ea80e18dc1429f953ec638694778b9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:44.740075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ea80e18dc1429f953ec638694778b9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:44.740693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ea80e18dc1429f953ec638694778b9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:44.753926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ea80e18dc1429f953ec638694778b9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:44.759327 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:44.768024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1337/2635] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:44 2022] DELETE /identity/v3/users/d4ea80e18dc1429f953ec638694778b9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:44.773164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.773656 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.773753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.794159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:44.795288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:44.865561 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2636] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:44.877760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:44.887126 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:44.962877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c496f-599e-46c6-a565-fac13b8da8dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:44.967041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:44.967721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:44.968039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:44.968331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:44.978773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:44.980772 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:44.992197 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1338/2637] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:45.566187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:45.568847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:45.607249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630c1ec-1375-4667-885e-1be319d250d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:45.613365 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:45.614297 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:45.614418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:45.614751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] PATH_INFO: `/v3/projects/1b2108031095477199883c48cfc39522` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:45.616131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2108031095477199883c48cfc39522)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:45.618709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:45.681928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2108031095477199883c48cfc39522', 'request_id': 'req-910997bc-a5bd-4337-83fc-e58e405b372c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.692279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2108031095477199883c48cfc39522', 'request_id': 'req-910997bc-a5bd-4337-83fc-e58e405b372c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.700576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2108031095477199883c48cfc39522', 'request_id': 'req-910997bc-a5bd-4337-83fc-e58e405b372c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.726139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] The token cache is being invalidate because project 1b2108031095477199883c48cfc39522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:45.727528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.728166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.731463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2638] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:45 2022] DELETE /identity/v3/projects/1b2108031095477199883c48cfc39522 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:45.738147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:45.773832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118c8e0-4fd9-43b9-8cc3-8d4977e48974 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:45.877661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:45.878331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:45.878702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:45.878875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] PATH_INFO: `/v3/users/a32bd01a35bb469a93a2bcf80ffade4b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:45.880632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32bd01a35bb469a93a2bcf80ffade4b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:45.882579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:45.958866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32bd01a35bb469a93a2bcf80ffade4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.970887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32bd01a35bb469a93a2bcf80ffade4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.971818 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32bd01a35bb469a93a2bcf80ffade4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.972540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32bd01a35bb469a93a2bcf80ffade4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:45.989693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c162739-755b-4cb2-bafc-972bef5c594a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32bd01a35bb469a93a2bcf80ffade4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:46.006811 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1339/2639] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:45 2022] DELETE /identity/v3/users/a32bd01a35bb469a93a2bcf80ffade4b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:46.458098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:46.460331 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:46.487927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910997bc-a5bd-4337-83fc-e58e405b372c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:46.574618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:46.575455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:46.575833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:46.576175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] PATH_INFO: `/v3/projects/3de3473157a54442822ac0c6ed2c6fc6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:46.577448 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de3473157a54442822ac0c6ed2c6fc6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:46.579504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:46.591800 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:46.639747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de3473157a54442822ac0c6ed2c6fc6', 'request_id': 'req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:46.648197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de3473157a54442822ac0c6ed2c6fc6', 'request_id': 'req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:46.655491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de3473157a54442822ac0c6ed2c6fc6', 'request_id': 'req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:46.672606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] The token cache is being invalidate because project 3de3473157a54442822ac0c6ed2c6fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:46.673034 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:46.673812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc4b55e2-903d-4611-9f4f-0926a1d5abbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:46.675713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2640] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:46 2022] DELETE /identity/v3/projects/3de3473157a54442822ac0c6ed2c6fc6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:46.681665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:46.682017 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:46.682279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:46.707594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:46.709104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:46.726101 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:46.780100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1340/2641] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:46.787815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:46.788185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:46.788536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:46.809649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:46.810888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:46.819149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2642] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:46.823844 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:46.833735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532916e6-6f8c-4207-afb6-b414a72417a8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:46.838428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:46.839800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:46.840169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:46.840449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:46.841154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:46.841462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:46.843104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:46.859419 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1341/2643] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:28:46.860457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:46.870483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91beb-38e0-4a5d-9579-147898dc52f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:46.875318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:46.876025 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:46.876317 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:46.876595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:46.877266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:46.879192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:46.917140 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2644] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:46 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:46.919429 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:46.937621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bf2ea-4628-4e90-acbc-a7637c49c719 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.029042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.029696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.029986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.030323 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.031896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.034359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.066762 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1342/2645] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:46 2022] POST /identity/v3/users => generated 282 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:47.068325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.089430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445ba7a-f35e-43b9-97d3-a8550d71c367 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.097443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.098234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.098547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.099385 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.099653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.101574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.117661 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2646] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:47.119783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.132482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315847b4-31da-4001-97bc-e165739df27a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.142730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b9c7a-e2c6-44b5-8d67-6ea79623dd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.143750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b9c7a-e2c6-44b5-8d67-6ea79623dd5d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.144109 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b9c7a-e2c6-44b5-8d67-6ea79623dd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.144457 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b9c7a-e2c6-44b5-8d67-6ea79623dd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.145321 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b9c7a-e2c6-44b5-8d67-6ea79623dd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.147074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b9c7a-e2c6-44b5-8d67-6ea79623dd5d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.163406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1343/2647] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:47.165938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.186248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3e29f-1739-4620-b208-f6aa33d09f90 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.192831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.193583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.194045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.194192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] PATH_INFO: `/v3/projects/f5e0922d0ed94665810727d27aaa75e4/users/2a49a2615a754846afad226e9d7d084c/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.215810 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e0922d0ed94665810727d27aaa75e4, user_id=2a49a2615a754846afad226e9d7d084c, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.218834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.236398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2648] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:47 2022] PUT /identity/v3/projects/f5e0922d0ed94665810727d27aaa75e4/users/2a49a2615a754846afad226e9d7d084c/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:47.248076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.248369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.248649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.290260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] MFA Rules not processed for user `2a49a2615a754846afad226e9d7d084c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:47.292330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:47.398050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1344/2649] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:47.403996 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.520329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2d89c-aa84-43dc-9b77-e5b00bfb7228 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.526707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.527613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.527980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.528504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.529257 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.533455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.593368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2650] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:47 2022] POST /identity/v3/projects => generated 363 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:47.597502 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.615174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1cac-d72a-46c0-b586-db039d5b8b9f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.642523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:47.700771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.701408 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.701697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.701979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.702661 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.704692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.731478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1345/2651] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:47 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:47.733203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.744688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eebab-c28a-4c99-8592-9e72439397ab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.750260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.750895 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.751182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.751498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.752197 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.754084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.767243 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2652] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:47.772515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.784230 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32dc24-00f5-47c3-b665-5595951764ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.789792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.790512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.790799 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.791080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] PATH_INFO: `/v3/projects/fb49cf0b4c0141199dae19544f5e2a4d/users/f31fe0a02ca546e1852a05d8257acc9e/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.812573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb49cf0b4c0141199dae19544f5e2a4d, user_id=f31fe0a02ca546e1852a05d8257acc9e, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.815370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.831282 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1346/2653] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:47 2022] PUT /identity/v3/projects/fb49cf0b4c0141199dae19544f5e2a4d/users/f31fe0a02ca546e1852a05d8257acc9e/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:47.835581 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.850969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0989-175a-4d0f-99ad-d387f599b3ea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.857045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:47.919710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c3550-a87d-4cdc-b566-640adf6c1d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.920258 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c3550-a87d-4cdc-b566-640adf6c1d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.920547 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c3550-a87d-4cdc-b566-640adf6c1d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.920827 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c3550-a87d-4cdc-b566-640adf6c1d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.921528 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c3550-a87d-4cdc-b566-640adf6c1d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.923135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c3550-a87d-4cdc-b566-640adf6c1d54 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.933684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2654] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:47.938192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:47.950525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d13a8-3955-4f82-b8a0-8b607b68b7e3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:47.955685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:47.956360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.956686 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.956952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] PATH_INFO: `/v3/projects/fb49cf0b4c0141199dae19544f5e2a4d/users/f31fe0a02ca546e1852a05d8257acc9e/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:47.958923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb49cf0b4c0141199dae19544f5e2a4d, user_id=f31fe0a02ca546e1852a05d8257acc9e, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:47.961610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:47.973165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1347/2655] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:47 2022] PUT /identity/v3/projects/fb49cf0b4c0141199dae19544f5e2a4d/users/f31fe0a02ca546e1852a05d8257acc9e/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:47.984567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:47.985000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:47.985291 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:48.020723 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] MFA Rules not processed for user `f31fe0a02ca546e1852a05d8257acc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:48.021956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:48.096028 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2656] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:47 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:48.110683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:48.113075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:48.185569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895fefe-c776-40d1-8a62-e7814f3f2e7e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:48.189341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:48.189964 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:48.190251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:48.190533 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:48.200719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:48.202185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:48.212188 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1348/2657] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:48.297203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:48.310134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834f446-a9a4-4a5b-821b-196a60caa72a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:48.314325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:48.314986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:48.315304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:48.315680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:48.379888 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:48.381329 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:48.390513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2658] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:28:50.264358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:50.265792 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:50.346583 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f5c11-4ded-4e7c-8006-45905b35f730 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:50.351768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:50.352465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:50.352681 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:50.352956 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:50.354745 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:50.381386 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:50.438160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:50.439823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f644206-4bfb-4a82-83a4-eeb1a5e261bc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:50.451663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1349/2659] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:54.022272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:54.036970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e76c4-73c4-40e2-8f7e-98522b378a08 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:54.043819 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:54.044655 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:54.044946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:54.045233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] PATH_INFO: `/v3/users/2a49a2615a754846afad226e9d7d084c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:54.046998 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a49a2615a754846afad226e9d7d084c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:54.049364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:54.082217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:54.144896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a49a2615a754846afad226e9d7d084c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:54.146255 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a49a2615a754846afad226e9d7d084c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:54.158358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a49a2615a754846afad226e9d7d084c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:54.159458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a49a2615a754846afad226e9d7d084c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:54.191043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a49a2615a754846afad226e9d7d084c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:54.231787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2660] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:54 2022] DELETE /identity/v3/users/2a49a2615a754846afad226e9d7d084c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:54.240221 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:54.241053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:54.241053 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:54.280646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:54.282526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:54.454697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1350/2661] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:54.466590 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:54.471105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:54.648002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c9a6e-31c1-4499-83e2-ef849e03523b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:54.664900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:54.666410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:54.667165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:54.667972 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:54.703010 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:54.706104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:54.731721 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2662] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:55.853650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:55.855732 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:55.874611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f618df-9695-40dd-8097-ae14dcc0f678 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:55.883708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:55.884807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:55.885384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:55.888288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] PATH_INFO: `/v3/projects/f5e0922d0ed94665810727d27aaa75e4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:55.890917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e0922d0ed94665810727d27aaa75e4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:55.893822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:55.995007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e0922d0ed94665810727d27aaa75e4', 'request_id': 'req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.015986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e0922d0ed94665810727d27aaa75e4', 'request_id': 'req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.037100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e0922d0ed94665810727d27aaa75e4', 'request_id': 'req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.060342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] The token cache is being invalidate because project f5e0922d0ed94665810727d27aaa75e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:56.061402 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.062777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.071747 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1351/2663] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:55 2022] DELETE /identity/v3/projects/f5e0922d0ed94665810727d27aaa75e4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:56.074322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:56.093871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbc640-93cd-459a-891a-9dd3ded974fc service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:56.231141 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:56.232131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:56.232749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:56.233318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] PATH_INFO: `/v3/users/f31fe0a02ca546e1852a05d8257acc9e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:56.235429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31fe0a02ca546e1852a05d8257acc9e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:56.239899 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:56.282478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:56.284263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:56.347987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59479f60-ff71-4f85-8b0e-0ef6c8414c4d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:56.368168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:56.369525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:56.370701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:56.375883 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] PATH_INFO: `/v3/users/af5a109a6fb94042aeacbf141d7322ef` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:56.381557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5a109a6fb94042aeacbf141d7322ef)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:56.387787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:56.451233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:56.470582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31fe0a02ca546e1852a05d8257acc9e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.477769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31fe0a02ca546e1852a05d8257acc9e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.551457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31fe0a02ca546e1852a05d8257acc9e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.553711 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31fe0a02ca546e1852a05d8257acc9e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.553930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:56.592638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5a109a6fb94042aeacbf141d7322ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.595988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31fe0a02ca546e1852a05d8257acc9e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.618160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5a109a6fb94042aeacbf141d7322ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.623304 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5a109a6fb94042aeacbf141d7322ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.624369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5a109a6fb94042aeacbf141d7322ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.626912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2664] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:56 2022] DELETE /identity/v3/users/f31fe0a02ca546e1852a05d8257acc9e => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:56.662442 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d95468a-288b-4a56-b4e6-057dc3e0b9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5a109a6fb94042aeacbf141d7322ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:56.686884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1352/2665] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:28:56 2022] DELETE /identity/v3/users/af5a109a6fb94042aeacbf141d7322ef => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:57.111040 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:57.113364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:57.269052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb335e1-84a8-433e-b554-9b74450156a7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:57.365795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:57.366483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:57.366779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:57.367062 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] PATH_INFO: `/v3/projects/9fda8c6d9ffa4955b2f19411ee340cb3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:57.368380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fda8c6d9ffa4955b2f19411ee340cb3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:57.370370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:57.575959 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fda8c6d9ffa4955b2f19411ee340cb3', 'request_id': 'req-2e383742-3d12-4437-9097-b1cb3b60d0ca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.587757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fda8c6d9ffa4955b2f19411ee340cb3', 'request_id': 'req-2e383742-3d12-4437-9097-b1cb3b60d0ca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.606227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fda8c6d9ffa4955b2f19411ee340cb3', 'request_id': 'req-2e383742-3d12-4437-9097-b1cb3b60d0ca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.629604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:57.639037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] The token cache is being invalidate because project 9fda8c6d9ffa4955b2f19411ee340cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:57.640598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.644724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.648328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2666] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:57 2022] DELETE /identity/v3/projects/9fda8c6d9ffa4955b2f19411ee340cb3 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:57.655079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ce60e-6013-407f-a1d1-8c3ba67d102d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:57.656585 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ce60e-6013-407f-a1d1-8c3ba67d102d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:57.656973 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ce60e-6013-407f-a1d1-8c3ba67d102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:57.688001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-800ce60e-6013-407f-a1d1-8c3ba67d102d None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:57.690432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-800ce60e-6013-407f-a1d1-8c3ba67d102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:57.719286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:57.721470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:57.819676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1353/2667] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:57.829065 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:57.829518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:57.829828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:57.856452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:57.857875 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:57.867393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1354/2668] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:57.873242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:57.877893 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383742-3d12-4437-9097-b1cb3b60d0ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:57.887725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b31b-9fd0-4097-b65d-ae0d5aa9628e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:57.893248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:57.894624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:57.894913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:57.895191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] PATH_INFO: `/v3/projects/fb49cf0b4c0141199dae19544f5e2a4d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:57.896535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb49cf0b4c0141199dae19544f5e2a4d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:57.898589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:57.898947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:57.899241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:57.899559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:57.899849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:57.900569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:57.900887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:57.903218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:57.916693 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1355/2669] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:28:57.921905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:57.939512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44b3c-3fed-43de-a17d-ba675d9ed010 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:57.950538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:57.951605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:57.952025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:57.952430 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:57.953566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:57.957579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:57.973381 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb49cf0b4c0141199dae19544f5e2a4d', 'request_id': 'req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.983979 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb49cf0b4c0141199dae19544f5e2a4d', 'request_id': 'req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:57.995507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb49cf0b4c0141199dae19544f5e2a4d', 'request_id': 'req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:58.011171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1356/2670] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:57 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:58.017423 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.026525 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] The token cache is being invalidate because project fb49cf0b4c0141199dae19544f5e2a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:28:58.026861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:58.031694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7e5dcca-eafb-423e-b7b6-9658ad1b7b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:28:58.036307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2671] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:28:57 2022] DELETE /identity/v3/projects/fb49cf0b4c0141199dae19544f5e2a4d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:58.042107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe822f0-1a79-4eb8-9cf5-7f43ed68cc5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.042748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe822f0-1a79-4eb8-9cf5-7f43ed68cc5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.045149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe822f0-1a79-4eb8-9cf5-7f43ed68cc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.089258 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4fe822f0-1a79-4eb8-9cf5-7f43ed68cc5b None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:58.092507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4fe822f0-1a79-4eb8-9cf5-7f43ed68cc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:58.127721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987552d-47fe-47c8-8d3b-d47c9abe1ada admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.137566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.138907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.139546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.140145 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.141557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.145812 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.180390 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2672] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:58.191184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.191808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.195014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1357/2673] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:58.195144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.199986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.201501 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:58.215020 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:58.222877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bca0b-47fc-4a09-8d33-578c3853687e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.231829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:58.232135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.232607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.232896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.233180 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.233922 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.234826 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:58.236277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.255659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2674] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:58.260488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.271082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1358/2675] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:58.275454 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.282226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7b2cb-e504-480b-bedb-f2b9cee3d121 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.300208 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.300964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.301298 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.301637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.302429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:28:58.302799 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.303097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c457881-b4a8-4fda-a9e6-c96653f414f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.304666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.311466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.312548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.313123 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.313698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.314920 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.319619 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.322080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2676] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:28:58.329264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.343054 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1359/2677] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:58.344276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.348822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc803-3988-4f98-8f8f-6c0ab765d31d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.361014 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ca61-3d21-427e-95f6-847bca7db567 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.364084 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.365278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.365839 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.366443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.368156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.369471 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a197e-cd3a-4cdb-af5c-36e44e7c74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.370381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a197e-cd3a-4cdb-af5c-36e44e7c74fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.371251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a197e-cd3a-4cdb-af5c-36e44e7c74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.372652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.372957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a197e-cd3a-4cdb-af5c-36e44e7c74fb admin admin] PATH_INFO: `/v3/projects/52a246d044bc45fcab65f1bce64f6a17/users/8ce32ade116943d0848e0e0142c5db27/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.392577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a197e-cd3a-4cdb-af5c-36e44e7c74fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a246d044bc45fcab65f1bce64f6a17, user_id=8ce32ade116943d0848e0e0142c5db27, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.394907 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:58.395396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a197e-cd3a-4cdb-af5c-36e44e7c74fb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.421104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1360/2678] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:58 2022] PUT /identity/v3/projects/52a246d044bc45fcab65f1bce64f6a17/users/8ce32ade116943d0848e0e0142c5db27/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:58.433385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.433621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2679] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:58.434100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.435317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.440229 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.493461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] MFA Rules not processed for user `8ce32ade116943d0848e0e0142c5db27`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:58.495849 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:58.550618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb78b3-5d46-4f01-8557-1efc8f0f7dfb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.557418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.558240 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.558639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.559642 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.560351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.562929 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.592066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2680] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:28:58.594788 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1361/2681] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:58.597387 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.598962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.609804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766215d5-6ca9-4382-b536-4bd8b08f82d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.614820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690ba58-f5ff-416a-a5f7-e3e944a88a3f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.618015 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.618668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.619014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.619272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.620074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.621988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.622741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.623081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.623267 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.623451 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.624183 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.626305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.644476 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2682] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:58.650576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.662212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93dbe4-71f5-4194-9161-6bb05ddc7c0d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.669467 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1362/2683] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:58.671011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.672049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.672548 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.673316 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.674276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.674463 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.676734 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.696804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2684] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:28:58.705227 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.727105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2a9ae-bea7-467a-92e7-611cac9b8faf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.785169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fff2b5-36e4-4bd9-9844-81b3fd65f346 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.804573 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.804848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.805095 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.805382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.806087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.806836 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e171ef-b3b3-47c5-9511-1f2caff280d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:58.808596 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e171ef-b3b3-47c5-9511-1f2caff280d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.809441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.809580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e171ef-b3b3-47c5-9511-1f2caff280d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.810391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e171ef-b3b3-47c5-9511-1f2caff280d6 admin admin] PATH_INFO: `/v3/projects/990d473a2c6e4602a60ca4364a725741/users/8e97fb4c84c54d4ea1bbd79721f0aab5/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.855082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e171ef-b3b3-47c5-9511-1f2caff280d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990d473a2c6e4602a60ca4364a725741, user_id=8e97fb4c84c54d4ea1bbd79721f0aab5, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:58.860910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e171ef-b3b3-47c5-9511-1f2caff280d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:58.867835 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1363/2685] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:28:58.874130 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:58.885553 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ec94-d025-4b87-bd6c-743534d772fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:58.887498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2686] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:58 2022] PUT /identity/v3/projects/990d473a2c6e4602a60ca4364a725741/users/8e97fb4c84c54d4ea1bbd79721f0aab5/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:58.894890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:58.895882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:58.897975 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:58.899241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:58.951494 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:58.953832 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:59.000954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:59.007315 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:59.010787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.010787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.010787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.017151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:59.020227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:59.051397 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2687] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:28:59.052155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1364/2688] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:59.052262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:59.052358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:59.067385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:59.067621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d1b4d-7192-4bd9-9baa-a42f9a71edbc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:59.087389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:59.087389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.087389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.087389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] PATH_INFO: `/v3/projects/c89e23c9a2fa4627a58f7c09787472a9/users/aef93f0e68a44e72966c147be81d4425/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.107500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89e23c9a2fa4627a58f7c09787472a9, user_id=aef93f0e68a44e72966c147be81d4425, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:59.107500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:59.131311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2689] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:59 2022] PUT /identity/v3/projects/c89e23c9a2fa4627a58f7c09787472a9/users/aef93f0e68a44e72966c147be81d4425/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:28:59.133393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:59.176568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bbc06-ae94-4a55-8e73-731896497a80 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:59.281007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bea2e0-a209-4ae4-b0d4-1fd08fc270df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:59.281659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea2e0-a209-4ae4-b0d4-1fd08fc270df service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.282104 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea2e0-a209-4ae4-b0d4-1fd08fc270df service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.282395 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea2e0-a209-4ae4-b0d4-1fd08fc270df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.282874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ef17e-af76-495e-beae-3ca4489042a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:59.287941 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:59.289402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.289402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.289402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.289954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:59.291648 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:59.294706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bea2e0-a209-4ae4-b0d4-1fd08fc270df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:59.296396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bea2e0-a209-4ae4-b0d4-1fd08fc270df service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:59.303398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2690] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:28:59.307573 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:59.319918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1365/2691] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:28:59.326503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf852c-6b89-49d5-9bc0-d34133cd4b6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:59.344206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:59.345152 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.345538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.345925 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] PATH_INFO: `/v3/projects/c89e23c9a2fa4627a58f7c09787472a9/users/aef93f0e68a44e72966c147be81d4425/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.348814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89e23c9a2fa4627a58f7c09787472a9, user_id=aef93f0e68a44e72966c147be81d4425, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:59.351846 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:59.366654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2692] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:28:59 2022] PUT /identity/v3/projects/c89e23c9a2fa4627a58f7c09787472a9/users/aef93f0e68a44e72966c147be81d4425/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:28:59.379828 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc76d8-c009-400a-8bf1-99f7199ce3d9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.380607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc76d8-c009-400a-8bf1-99f7199ce3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.380765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc76d8-c009-400a-8bf1-99f7199ce3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.423257 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-14fc76d8-c009-400a-8bf1-99f7199ce3d9 None None] MFA Rules not processed for user `aef93f0e68a44e72966c147be81d4425`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:28:59.426116 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-14fc76d8-c009-400a-8bf1-99f7199ce3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:59.536687 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:59.555280 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1366/2693] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:28:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:28:59.571703 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:28:59.575477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:28:59.696963 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bcea4-dfb6-4c09-8344-d4cffb07319b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:28:59.703818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:28:59.704708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:28:59.705043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:28:59.705389 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:28:59.835414 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:28:59.842507 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:28:59.844314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:28:59.868210 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2694] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:28:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:00.477033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:00.477790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:00.477790 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:00.513890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] MFA Rules not processed for user `8e97fb4c84c54d4ea1bbd79721f0aab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:00.515261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:00.641580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:00.643670 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1367/2695] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:00.741106 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f148-d099-474b-8d10-1b1226d16516 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:00.746511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:00.747262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:00.747584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:00.747840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:00.748517 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:00.750457 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:00.782675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2696] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:00 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:00.787262 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:00.799032 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff7448-746a-4b70-bf16-7f28ed25c48d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:00.877540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:00.878492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:00.878794 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:00.879150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:00.879960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:00.882215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:00.910295 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1368/2697] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:00 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:00.914958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:00.934911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db48b-2a8f-425f-9157-3ad72408b858 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:00.939128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:00.939812 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:00.940103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:00.940384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:00.941099 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:00.943148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:00.959466 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2698] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:00.963924 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:00.983264 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9010bc8-4d61-471b-8606-816fc4829779 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:00.989837 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:00.990569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:00.990906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:00.991195 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] PATH_INFO: `/v3/projects/95b0171d16cf43f6a5590c86266ad9c2/users/ffbdfb96c9bd45f58afe8ad880537a2c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:01.021631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b0171d16cf43f6a5590c86266ad9c2, user_id=ffbdfb96c9bd45f58afe8ad880537a2c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:01.028631 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:01.050161 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1369/2699] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:00 2022] PUT /identity/v3/projects/95b0171d16cf43f6a5590c86266ad9c2/users/ffbdfb96c9bd45f58afe8ad880537a2c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:01.054434 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:01.066430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252688a-11a5-44f7-bb95-a3e78b7c1c93 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:01.135393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa21890-9d82-4e8b-a7bc-d274f9b97d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:01.136038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa21890-9d82-4e8b-a7bc-d274f9b97d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:01.136321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa21890-9d82-4e8b-a7bc-d274f9b97d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:01.136688 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa21890-9d82-4e8b-a7bc-d274f9b97d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:01.137613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa21890-9d82-4e8b-a7bc-d274f9b97d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:01.139236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa21890-9d82-4e8b-a7bc-d274f9b97d7a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:01.150493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2700] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:29:01.155925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:01.166636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3e277-26ab-416e-aa3f-bb2ac4a3636f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:01.173096 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:01.173970 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:01.174434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:01.174984 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] PATH_INFO: `/v3/projects/95b0171d16cf43f6a5590c86266ad9c2/users/ffbdfb96c9bd45f58afe8ad880537a2c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:01.177862 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b0171d16cf43f6a5590c86266ad9c2, user_id=ffbdfb96c9bd45f58afe8ad880537a2c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:01.181506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:01.193845 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1370/2701] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:01 2022] PUT /identity/v3/projects/95b0171d16cf43f6a5590c86266ad9c2/users/ffbdfb96c9bd45f58afe8ad880537a2c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:02.095849 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:02.096373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:02.096505 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:02.129911 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] MFA Rules not processed for user `ffbdfb96c9bd45f58afe8ad880537a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:02.131234 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:02.220920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:02.257137 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2702] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:02.277910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:02.281660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:02.349209 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:02.384228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e915d-20ad-4ebe-8fd6-cfefc4f02869 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:02.391925 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:02.392961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:02.393421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:02.393897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:02.487267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:02.489915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:02.507460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1371/2703] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:29:12.252813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:12.499816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=32 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:13.442220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:13.460000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4600b-f280-4b58-b983-9b83d07f2b3f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:13.466319 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:13.467509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:13.468045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:13.468437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:13.576517 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:13.584795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:13.586647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dd4c2-fab3-480e-acce-9bccc2a22d5d service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:13.586903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:13.591652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15235f4-9280-4a89-a210-a317fc984125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:13.592624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15235f4-9280-4a89-a210-a317fc984125 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:13.593150 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15235f4-9280-4a89-a210-a317fc984125 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:13.593632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15235f4-9280-4a89-a210-a317fc984125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:13.603337 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2704] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:29:13.605760 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15235f4-9280-4a89-a210-a317fc984125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:13.607431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15235f4-9280-4a89-a210-a317fc984125 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:13.619191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1372/2705] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:29:14.550510 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:14.552803 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:14.684396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e95b-24b0-4d9a-9141-aea150220293 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:14.689178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:14.689873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:14.690235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:14.690521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] PATH_INFO: `/v3/users/b249dd9d75504505ad67fa100fc3cdf7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:14.691873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b249dd9d75504505ad67fa100fc3cdf7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:14.695976 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:14.755052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b249dd9d75504505ad67fa100fc3cdf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:14.756272 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b249dd9d75504505ad67fa100fc3cdf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:14.765134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b249dd9d75504505ad67fa100fc3cdf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:14.765891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b249dd9d75504505ad67fa100fc3cdf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:14.783328 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b249dd9d75504505ad67fa100fc3cdf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:14.803168 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2706] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:14 2022] DELETE /identity/v3/users/b249dd9d75504505ad67fa100fc3cdf7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:14.810896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:14.811520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:14.811972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:14.835940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:14.837405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:14.909564 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1373/2707] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:14.924309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:14.925766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:14.942074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:14.994351 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e448fb-b796-48dc-8bc1-57e453acaf31 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:15.000052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11992453-a7df-4cc2-99ed-e36c429cdf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:15.000878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11992453-a7df-4cc2-99ed-e36c429cdf80 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.001292 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11992453-a7df-4cc2-99ed-e36c429cdf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.001725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11992453-a7df-4cc2-99ed-e36c429cdf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.013861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11992453-a7df-4cc2-99ed-e36c429cdf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:15.015341 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11992453-a7df-4cc2-99ed-e36c429cdf80 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:15.028537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2708] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:15.406415 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:15.408449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:15.431941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c506873-f430-469a-a0d5-f4d82a2a433e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:15.452446 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:15.453281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.453775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.454309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] PATH_INFO: `/v3/projects/a3c16e58753c41149a3f8f2ef4ce1b0d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.456796 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c16e58753c41149a3f8f2ef4ce1b0d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:15.460697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:15.558855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c16e58753c41149a3f8f2ef4ce1b0d', 'request_id': 'req-7afd7811-6f01-4e13-8ff7-1256e333386a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:15.571060 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c16e58753c41149a3f8f2ef4ce1b0d', 'request_id': 'req-7afd7811-6f01-4e13-8ff7-1256e333386a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:15.580653 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c16e58753c41149a3f8f2ef4ce1b0d', 'request_id': 'req-7afd7811-6f01-4e13-8ff7-1256e333386a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:15.608254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] The token cache is being invalidate because project a3c16e58753c41149a3f8f2ef4ce1b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:15.608254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:15.608254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7afd7811-6f01-4e13-8ff7-1256e333386a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:15.608254 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1374/2709] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:15 2022] DELETE /identity/v3/projects/a3c16e58753c41149a3f8f2ef4ce1b0d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:15.616579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.617236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.617483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.643177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:15.644591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:15.728169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2710] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:15.738378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.738668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.739106 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.764080 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:15.765334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:15.774059 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1375/2711] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:15.779739 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:15.790384 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83831b3-f99c-4020-9f9b-b76ca009a8e1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:15.796048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:15.796764 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.797707 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.797997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.798729 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:15.799042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:15.800780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:15.817900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2712] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:29:15.822466 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:15.832158 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8ce36-9727-4392-bbb8-f7df75d77aad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:15.837218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:15.837859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.838146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.838428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.839703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:15.841669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:15.884556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1376/2713] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:15 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:15.885820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:15.896537 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f916b-045e-4e0c-a04c-5f28c94746cd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:15.963085 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:15.963933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:15.964268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:15.964532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:15.965307 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:15.971043 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:16.009571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2714] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:15 2022] POST /identity/v3/users => generated 281 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:16.016314 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:16.032277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6595ae-74bb-4a96-8889-5704d5113031 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:16.038539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:16.039482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:16.039976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:16.040463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:16.041742 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:16.044860 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:16.063291 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1377/2715] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:16.067264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:16.078222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5134a-00c3-4692-9bf9-109fd9dec6b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:16.083223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0a853-9223-485b-8b7d-35b0bf8e11fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:16.083880 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a853-9223-485b-8b7d-35b0bf8e11fc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:16.084220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a853-9223-485b-8b7d-35b0bf8e11fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:16.084500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a853-9223-485b-8b7d-35b0bf8e11fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:16.085231 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0a853-9223-485b-8b7d-35b0bf8e11fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:16.086829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0a853-9223-485b-8b7d-35b0bf8e11fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:16.098877 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2716] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:16.103513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:16.115134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fe05f-12c9-48ce-8307-f51bb3ec2e1e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:16.120765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:16.121469 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:16.121756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:16.122038 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] PATH_INFO: `/v3/projects/409bdfebeee241389844e3015aab0a86/users/a5546058044b433d9907bea616cc9aa8/roles/f5c39945c9694134955532988101506f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:16.139953 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] RBAC: Authorizing `identity:create_grant(project_id=409bdfebeee241389844e3015aab0a86, user_id=a5546058044b433d9907bea616cc9aa8, role_id=f5c39945c9694134955532988101506f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:16.142665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:16.158002 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1378/2717] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:16 2022] PUT /identity/v3/projects/409bdfebeee241389844e3015aab0a86/users/a5546058044b433d9907bea616cc9aa8/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:16.169668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:16.170100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:16.170535 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:16.201360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] MFA Rules not processed for user `a5546058044b433d9907bea616cc9aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:16.202595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:16.217779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:16.283969 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2718] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:16.296689 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:16.298444 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:16.309022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:16.373010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16d2b-152c-444d-9ac6-a1cafb60489b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:16.376931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:16.377591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:16.377876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:16.378154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:16.388947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:16.390594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:16.402369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1379/2719] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:18.726336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:18.728020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:18.852269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d92b46-52a4-4e8e-8ce7-85cef572202a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:18.861131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:18.862496 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:18.863796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:18.863796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:18.867154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:18.882402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:18.884568 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe80c-a6cd-4365-a2e4-b4d4c8d110a9 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:18.899216 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2720] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:24.911559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:24.913178 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:24.982907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbfb0e-671f-4c2c-8ec9-8f953ad4f5a3 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:24.986703 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:24.987341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:24.987656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:24.987936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:24.989868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:25.063580 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:25.065061 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:25.077735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1380/2721] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:27.949120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:27.949994 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:27.950181 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:27.992700 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] MFA Rules not processed for user `8d737070d3404633a4f6fdd25caa8964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:27.996274 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:28.113176 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2722] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 05:29:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:28.185222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:28.188008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:28.310306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:28.320372 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e902b8-0849-4b44-94c5-6146d2fb45be service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:28.329679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1176e3df3023409b93375dc94ffb661c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:28.330342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:28.330627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:28.331102 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:28.344975 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:28.347699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:28.363125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1381/2723] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 05:29:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 05:29:37.640020 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:37.642237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:37.684483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:37.737012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f622b-1343-4174-8492-d897d729f5e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:37.742118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:37.742913 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:37.743890 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:37.744302 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] PATH_INFO: `/v3/users/190f6130be544a73bb619f7b49ab4e87` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:37.745697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190f6130be544a73bb619f7b49ab4e87)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:37.748048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:37.818407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190f6130be544a73bb619f7b49ab4e87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:37.819755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190f6130be544a73bb619f7b49ab4e87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:37.835256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190f6130be544a73bb619f7b49ab4e87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:37.836490 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190f6130be544a73bb619f7b49ab4e87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:37.861205 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190f6130be544a73bb619f7b49ab4e87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:37.881611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2724] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:37 2022] DELETE /identity/v3/users/190f6130be544a73bb619f7b49ab4e87 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:38.196008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:38.231934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300248cc-5d54-44ea-b50d-34c77789d4d8 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:38.372369 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:38.373519 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:38.373977 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:38.374488 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] PATH_INFO: `/v3/projects/4c28b78169ac49a6854f195bea0dd9bf` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:38.377902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c28b78169ac49a6854f195bea0dd9bf)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:38.384947 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:38.538873 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c28b78169ac49a6854f195bea0dd9bf', 'request_id': 'req-0ad40059-3234-459b-b134-2c3aaafcbba0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.559275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c28b78169ac49a6854f195bea0dd9bf', 'request_id': 'req-0ad40059-3234-459b-b134-2c3aaafcbba0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.569923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c28b78169ac49a6854f195bea0dd9bf', 'request_id': 'req-0ad40059-3234-459b-b134-2c3aaafcbba0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.586791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] The token cache is being invalidate because project 4c28b78169ac49a6854f195bea0dd9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:38.592247 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.593652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.595691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1382/2725] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:38 2022] DELETE /identity/v3/projects/4c28b78169ac49a6854f195bea0dd9bf => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:38.600970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:38.607089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:38.631935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35224-c22d-4101-853b-d47f7ab16265 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:38.741105 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:38.741804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:38.742031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:38.742309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] PATH_INFO: `/v3/users/1febec84e89b43cd8af72a7ed01e5e68` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:38.743881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1febec84e89b43cd8af72a7ed01e5e68)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:38.746206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:38.809910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:38.817482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1febec84e89b43cd8af72a7ed01e5e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.818779 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1febec84e89b43cd8af72a7ed01e5e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.839212 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1febec84e89b43cd8af72a7ed01e5e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.841289 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1febec84e89b43cd8af72a7ed01e5e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.868562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1febec84e89b43cd8af72a7ed01e5e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:38.913445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2726] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:38 2022] DELETE /identity/v3/users/1febec84e89b43cd8af72a7ed01e5e68 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:39.195140 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:39.207034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:39.226877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad40059-3234-459b-b134-2c3aaafcbba0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:39.326521 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:39.327373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:39.327668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:39.327952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] PATH_INFO: `/v3/projects/5f45c39eb99549d58fd35a248f6cbac6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:39.329179 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f45c39eb99549d58fd35a248f6cbac6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:39.331238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:39.363493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:39.403919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f45c39eb99549d58fd35a248f6cbac6', 'request_id': 'req-b6e864f6-5681-450c-8e83-adac1912e3b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.413943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f45c39eb99549d58fd35a248f6cbac6', 'request_id': 'req-b6e864f6-5681-450c-8e83-adac1912e3b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.425005 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f45c39eb99549d58fd35a248f6cbac6', 'request_id': 'req-b6e864f6-5681-450c-8e83-adac1912e3b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.451906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] The token cache is being invalidate because project 5f45c39eb99549d58fd35a248f6cbac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:39.452628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.453993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.457039 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1383/2727] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:39 2022] DELETE /identity/v3/projects/5f45c39eb99549d58fd35a248f6cbac6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:39.741677 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:39.744821 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:39.768509 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f90d37-a98f-45a6-8c42-2fa0c4f04ec7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:39.842479 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:39.843163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:39.843483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:39.843869 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] PATH_INFO: `/v3/users/8e97fb4c84c54d4ea1bbd79721f0aab5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:39.845064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e97fb4c84c54d4ea1bbd79721f0aab5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:39.847149 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:39.906207 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e97fb4c84c54d4ea1bbd79721f0aab5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.906882 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e97fb4c84c54d4ea1bbd79721f0aab5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.914312 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e97fb4c84c54d4ea1bbd79721f0aab5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.915042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e97fb4c84c54d4ea1bbd79721f0aab5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.930486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e97fb4c84c54d4ea1bbd79721f0aab5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:39.948544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2728] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:39 2022] DELETE /identity/v3/users/8e97fb4c84c54d4ea1bbd79721f0aab5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:40.194037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:40.196261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:40.213672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e864f6-5681-450c-8e83-adac1912e3b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:40.288678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:40.289411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:40.289702 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:40.289985 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] PATH_INFO: `/v3/projects/990d473a2c6e4602a60ca4364a725741` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:40.291988 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=990d473a2c6e4602a60ca4364a725741)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:40.294008 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:40.351306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990d473a2c6e4602a60ca4364a725741', 'request_id': 'req-09bda5ba-8f22-4e20-b67e-cb185712fc4e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.360523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990d473a2c6e4602a60ca4364a725741', 'request_id': 'req-09bda5ba-8f22-4e20-b67e-cb185712fc4e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.367589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990d473a2c6e4602a60ca4364a725741', 'request_id': 'req-09bda5ba-8f22-4e20-b67e-cb185712fc4e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.382483 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] The token cache is being invalidate because project 990d473a2c6e4602a60ca4364a725741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:40.382777 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.383495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.385561 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1384/2729] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:40 2022] DELETE /identity/v3/projects/990d473a2c6e4602a60ca4364a725741 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:40.389780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:40.391218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:40.405359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dff65-9311-4fb5-a634-4e023fdf3bb8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:40.444046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:40.477363 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:40.479031 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:40.479163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:40.479458 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] PATH_INFO: `/v3/users/ffbdfb96c9bd45f58afe8ad880537a2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:40.480710 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbdfb96c9bd45f58afe8ad880537a2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:40.482665 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:40.544474 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbdfb96c9bd45f58afe8ad880537a2c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.545134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbdfb96c9bd45f58afe8ad880537a2c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.553820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbdfb96c9bd45f58afe8ad880537a2c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.554539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbdfb96c9bd45f58afe8ad880537a2c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.568118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbdfb96c9bd45f58afe8ad880537a2c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.585138 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2730] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:40 2022] DELETE /identity/v3/users/ffbdfb96c9bd45f58afe8ad880537a2c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:40.684453 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:40.685934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:40.704924 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bda5ba-8f22-4e20-b67e-cb185712fc4e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:40.778602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:40.808427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:40.809199 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:40.809556 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:40.809994 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] PATH_INFO: `/v3/users/8ce32ade116943d0848e0e0142c5db27` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:40.811568 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce32ade116943d0848e0e0142c5db27)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:40.813551 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:40.817131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:40.819237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:40.853012 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dec8f-aad6-4923-9cc8-793e69bba176 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:40.859134 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:40.859843 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:40.860116 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:40.860591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] PATH_INFO: `/v3/projects/95b0171d16cf43f6a5590c86266ad9c2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:40.861833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b0171d16cf43f6a5590c86266ad9c2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:40.865032 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:40.907474 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce32ade116943d0848e0e0142c5db27'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.920696 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce32ade116943d0848e0e0142c5db27'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.921528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce32ade116943d0848e0e0142c5db27'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.922263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce32ade116943d0848e0e0142c5db27'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.939017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b0171d16cf43f6a5590c86266ad9c2', 'request_id': 'req-1ea10432-d9f8-4d08-9ad6-5d70671220fd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.942610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aa89dd33-8c78-4ab1-b8ef-ceda0600705e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce32ade116943d0848e0e0142c5db27'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.962813 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b0171d16cf43f6a5590c86266ad9c2', 'request_id': 'req-1ea10432-d9f8-4d08-9ad6-5d70671220fd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.970969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1385/2731] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:40 2022] DELETE /identity/v3/users/8ce32ade116943d0848e0e0142c5db27 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:40.975914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b0171d16cf43f6a5590c86266ad9c2', 'request_id': 'req-1ea10432-d9f8-4d08-9ad6-5d70671220fd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.976526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bccdc-5277-4613-b642-387877b7d482 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:40.976919 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bccdc-5277-4613-b642-387877b7d482 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:40.977349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bccdc-5277-4613-b642-387877b7d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:40.995093 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] The token cache is being invalidate because project 95b0171d16cf43f6a5590c86266ad9c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:40.995634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.996708 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ea10432-d9f8-4d08-9ad6-5d70671220fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:40.998835 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2732] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:40 2022] DELETE /identity/v3/projects/95b0171d16cf43f6a5590c86266ad9c2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:41.006393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-331bccdc-5277-4613-b642-387877b7d482 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:41.006940 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929a6c0-5d0b-4505-ab9e-913bef6f9d17 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.007668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929a6c0-5d0b-4505-ab9e-913bef6f9d17 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.008054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929a6c0-5d0b-4505-ab9e-913bef6f9d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.008343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-331bccdc-5277-4613-b642-387877b7d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.041530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7929a6c0-5d0b-4505-ab9e-913bef6f9d17 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:41.043478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7929a6c0-5d0b-4505-ab9e-913bef6f9d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.096865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1386/2733] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:29:41.109025 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bccdc-5277-4613-b642-387877b7d482 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.110763 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-331bccdc-5277-4613-b642-387877b7d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.111017 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:41.163955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2734] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:41.173876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.174241 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.174605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.216280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:41.216280 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.225288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2735] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:41.231905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.242341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bccdc-5277-4613-b642-387877b7d482 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.245552 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26191f-99bb-4b82-a635-c2b4fa2b94d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.246547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.247464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.247850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.248382 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.257611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.258308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.258634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.259804 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.261358 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:41.261511 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.263282 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.266827 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.269176 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.287885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2736] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:29:41.288097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1387/2737] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:41.290111 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.307838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac94fa0-0ea7-472c-b07f-71d9cb848e8d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.314336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.315041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.315383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.315684 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.316353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.318435 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.351406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2738] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:41 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:41.356317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.361669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:41.377830 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf616f1-beb0-4ed6-8af2-8461bd9c3e54 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.460330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.461003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.461293 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.461575 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.462877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.465288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.492070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1388/2739] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:41 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:29:41.498639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.524165 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aaa98-d265-4915-a9cc-38dcc7648ab7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.532847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.534068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.534468 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.535033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.536443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.539679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.562715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2740] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:41.568579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.580169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335373c-2e1f-4220-8e7b-7753320eaaef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.585261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.585952 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.586238 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.586528 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.588206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.589697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.601655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1389/2741] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:41.607036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.618818 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d6f54-ad16-488b-b9c6-3ec5c8734da7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.624112 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abfebf-39d9-4f8e-a85b-81147c0ee795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.624817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abfebf-39d9-4f8e-a85b-81147c0ee795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.625102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abfebf-39d9-4f8e-a85b-81147c0ee795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.625378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abfebf-39d9-4f8e-a85b-81147c0ee795 admin admin] PATH_INFO: `/v3/projects/b8edec2f81254cc7ad3fcbeb4e9c0315/users/a1076759a2a44b4b9cff5331484a29a6/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.647579 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abfebf-39d9-4f8e-a85b-81147c0ee795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8edec2f81254cc7ad3fcbeb4e9c0315, user_id=a1076759a2a44b4b9cff5331484a29a6, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.653937 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abfebf-39d9-4f8e-a85b-81147c0ee795 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.666119 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.668351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.695516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2742] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:41 2022] PUT /identity/v3/projects/b8edec2f81254cc7ad3fcbeb4e9c0315/users/a1076759a2a44b4b9cff5331484a29a6/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:41.700627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609cb95-056a-47f7-a7a6-f96f67a9972b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.712633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.714102 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.714470 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.756133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] MFA Rules not processed for user `a1076759a2a44b4b9cff5331484a29a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:41.757471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.804074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:41.804848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:41.805203 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:41.805636 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] PATH_INFO: `/v3/projects/52a246d044bc45fcab65f1bce64f6a17` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:41.807632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a246d044bc45fcab65f1bce64f6a17)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:41.809698 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:41.854816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2743] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:29:41.876396 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.878630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.891542 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a246d044bc45fcab65f1bce64f6a17', 'request_id': 'req-ffb07434-c9b2-492c-b487-6e2cbeba5239'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:41.900630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a246d044bc45fcab65f1bce64f6a17', 'request_id': 'req-ffb07434-c9b2-492c-b487-6e2cbeba5239'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:41.907006 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a246d044bc45fcab65f1bce64f6a17', 'request_id': 'req-ffb07434-c9b2-492c-b487-6e2cbeba5239'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:41.920421 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] The token cache is being invalidate because project 52a246d044bc45fcab65f1bce64f6a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:41.920716 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:41.921756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:41.923366 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1390/2744] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:41 2022] DELETE /identity/v3/projects/52a246d044bc45fcab65f1bce64f6a17 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:29:41.927714 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:41.930093 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:41.946228 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:41.975801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d6f4-b9e2-4d69-b590-0c14ffdc940f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:41.981861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:42.030648 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb07434-c9b2-492c-b487-6e2cbeba5239 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:42.044837 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:42.045491 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.045772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.046056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.046717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:42.047452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.047739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.048019 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] PATH_INFO: `/v3/users/aef93f0e68a44e72966c147be81d4425` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.048189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:42.049358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef93f0e68a44e72966c147be81d4425)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:42.051627 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:42.115647 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef93f0e68a44e72966c147be81d4425'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.125381 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef93f0e68a44e72966c147be81d4425'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.126151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef93f0e68a44e72966c147be81d4425'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.126822 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef93f0e68a44e72966c147be81d4425'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.137060 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:42.138557 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:42.145851 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fd8e038e-812e-40dc-954c-3a7a4ea3e75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef93f0e68a44e72966c147be81d4425'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.148792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2745] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:29:42.171757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1391/2746] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:41 2022] DELETE /identity/v3/users/aef93f0e68a44e72966c147be81d4425 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:29:42.559584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:42.583761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9e18f-1e2a-4d0f-aaa6-637271986cb9 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:42.662042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:42.662935 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.663222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.663538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] PATH_INFO: `/v3/projects/c89e23c9a2fa4627a58f7c09787472a9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.664774 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89e23c9a2fa4627a58f7c09787472a9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:42.667327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:42.720831 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89e23c9a2fa4627a58f7c09787472a9', 'request_id': 'req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.727266 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89e23c9a2fa4627a58f7c09787472a9', 'request_id': 'req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.733691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89e23c9a2fa4627a58f7c09787472a9', 'request_id': 'req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.736486 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:42.750753 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] The token cache is being invalidate because project c89e23c9a2fa4627a58f7c09787472a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:42.751317 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.752295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8c7e079-f1a2-406e-8ee2-9fce06bb905d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:42.754188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2747] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:42 2022] DELETE /identity/v3/projects/c89e23c9a2fa4627a58f7c09787472a9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:42.759697 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.760245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.760245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.786132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:42.787413 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:42.873131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1392/2748] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:29:42.885041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.886007 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.886306 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.919089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:42.920676 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:42.929695 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2749] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:42.934477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:42.946765 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7241311-9912-4b7d-8899-06a7485638bd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:42.952398 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:42.953051 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.953341 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.953624 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.954342 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:42.954655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:42.956394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:42.970422 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1393/2750] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:29:42.976113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:42.988693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd61f9-eeae-4393-87e9-85d93d077910 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:42.994204 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:42.994894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:42.995221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:42.995619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:42.996356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:42.999158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.043083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2751] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:42 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:43.047740 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.059026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ee6f-c811-49cb-aa55-1434d264eadf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.074655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:43.132773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.133574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.133986 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.134400 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.135460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.137598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.165935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1394/2752] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:43 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:29:43.171188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.183286 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b6536-c8a1-402b-a4ad-3b5cc93fac2f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.188504 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.189154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.189438 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.189720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.190431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.192132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.207418 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2753] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:29:43.213146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.224478 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e947a-0338-4a80-bb5d-2c1e690b3be5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.230712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfacfe-ebd4-4b3c-b3bd-1af734079e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.231384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfacfe-ebd4-4b3c-b3bd-1af734079e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.231678 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfacfe-ebd4-4b3c-b3bd-1af734079e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.232160 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfacfe-ebd4-4b3c-b3bd-1af734079e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.232882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfacfe-ebd4-4b3c-b3bd-1af734079e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.234506 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfacfe-ebd4-4b3c-b3bd-1af734079e8e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.247251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1395/2754] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:43.252221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.263796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d3ff-4c4c-48bc-b54b-b7cbc28b478b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.268876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.269576 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.269864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.270148 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] PATH_INFO: `/v3/projects/8280e7a952d347c3b54d8a1d49896ca2/users/633c0554f33842af9a9b9278dd21726d/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.288199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8280e7a952d347c3b54d8a1d49896ca2, user_id=633c0554f33842af9a9b9278dd21726d, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.291612 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.307939 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2755] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:43 2022] PUT /identity/v3/projects/8280e7a952d347c3b54d8a1d49896ca2/users/633c0554f33842af9a9b9278dd21726d/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:43.328050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.328536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.329100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.364212 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] MFA Rules not processed for user `633c0554f33842af9a9b9278dd21726d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:43.365535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:43.456549 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1396/2756] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:43.457674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.534521 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d58c02-f101-42b4-babc-912425496626 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.539378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.541052 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.541442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.541717 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.542391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.544339 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.580621 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2757] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:43 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:43.586443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.608866 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d9def-9324-4e01-ad16-cb15c9ee0c5d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.695010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.695815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.696782 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.697268 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.699022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.703204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.738761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1397/2758] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:43 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:29:43.742584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.755545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1a01-5aab-4006-a6bc-fbfdf4df48d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.764808 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.765767 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.766362 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.766938 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.768026 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.771185 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.786999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2759] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:29:43.792391 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.804306 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad46ff-1402-4182-b45a-ec8a006c6c6d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.811152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.812265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.812737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.813132 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] PATH_INFO: `/v3/projects/d62eb8f356b0471caf9e792a9a8d99fe/users/206b0c60b13749aa9184f2bb8b7000d3/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.840969 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:43.848010 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62eb8f356b0471caf9e792a9a8d99fe, user_id=206b0c60b13749aa9184f2bb8b7000d3, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.852165 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:43.872982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1398/2760] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:43 2022] PUT /identity/v3/projects/d62eb8f356b0471caf9e792a9a8d99fe/users/206b0c60b13749aa9184f2bb8b7000d3/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:43.879413 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:43.899124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33e3cc-4d4e-4609-b882-34126467091e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:43.987128 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad22af-4ab8-4c55-9ef4-b00a3dffc09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:43.987856 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad22af-4ab8-4c55-9ef4-b00a3dffc09c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:43.988188 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad22af-4ab8-4c55-9ef4-b00a3dffc09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:43.988488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad22af-4ab8-4c55-9ef4-b00a3dffc09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:43.989260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad22af-4ab8-4c55-9ef4-b00a3dffc09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:43.991054 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad22af-4ab8-4c55-9ef4-b00a3dffc09c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:44.001637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:44.012838 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2761] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:44.014028 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:44.034905 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3c0d-bee6-4688-8e3b-7c96162de3ab admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:44.046149 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:44.046971 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:44.047281 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:44.047646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] PATH_INFO: `/v3/projects/d62eb8f356b0471caf9e792a9a8d99fe/users/206b0c60b13749aa9184f2bb8b7000d3/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:44.064063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62eb8f356b0471caf9e792a9a8d99fe, user_id=206b0c60b13749aa9184f2bb8b7000d3, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:44.067731 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:44.080043 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1399/2762] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:44 2022] PUT /identity/v3/projects/d62eb8f356b0471caf9e792a9a8d99fe/users/206b0c60b13749aa9184f2bb8b7000d3/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:44.094731 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:44.095498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:44.096232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:44.152395 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] MFA Rules not processed for user `206b0c60b13749aa9184f2bb8b7000d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:44.154508 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:44.291368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2763] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:44.308664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:44.310810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:44.414552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1492d0-624c-4c4b-a69e-9a3d1c7d164f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:44.414552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:44.414552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:44.414552 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:44.415418 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:44.423539 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:44.425023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:44.437482 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1400/2764] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:44.504806 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:44.522724 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07be2c-0389-459c-9ba9-ec65550c5689 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:44.529775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdcc44-0f3f-4bfd-9e7f-da239da6a9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:44.530704 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcc44-0f3f-4bfd-9e7f-da239da6a9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:44.530876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcc44-0f3f-4bfd-9e7f-da239da6a9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:44.531122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcc44-0f3f-4bfd-9e7f-da239da6a9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:44.631404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcc44-0f3f-4bfd-9e7f-da239da6a9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:44.632881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcc44-0f3f-4bfd-9e7f-da239da6a9f3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:44.643430 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2765] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:45.615113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:45.616576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:45.686821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3842b6-ed61-4ee3-8333-6aa10eb218eb service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:45.691567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:45.692546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:45.692546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:45.692816 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] PATH_INFO: `/v3/users/a1076759a2a44b4b9cff5331484a29a6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:45.694026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1076759a2a44b4b9cff5331484a29a6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:45.696595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:45.750663 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1076759a2a44b4b9cff5331484a29a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:45.761033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1076759a2a44b4b9cff5331484a29a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:45.761708 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1076759a2a44b4b9cff5331484a29a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:45.762336 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1076759a2a44b4b9cff5331484a29a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:45.777087 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1076759a2a44b4b9cff5331484a29a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:45.786012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:45.790951 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1401/2766] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:45 2022] DELETE /identity/v3/users/a1076759a2a44b4b9cff5331484a29a6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:45.795942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:45.796565 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:45.796775 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:45.820036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:45.822079 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:45.889598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2767] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:45.900852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:45.902630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:45.968966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468db1d3-3c61-43c7-ba86-b7dceed311fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:45.972547 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ceccf-9f60-461f-99d2-7bd0b453d920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:45.973186 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ceccf-9f60-461f-99d2-7bd0b453d920 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:45.973468 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ceccf-9f60-461f-99d2-7bd0b453d920 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:45.973743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ceccf-9f60-461f-99d2-7bd0b453d920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:45.985276 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ceccf-9f60-461f-99d2-7bd0b453d920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:45.987011 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ceccf-9f60-461f-99d2-7bd0b453d920 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:45.999049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1402/2768] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:46.331916 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.333379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:46.364288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406aa6b-b33a-4496-9adb-63ab8ce6a4f9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.369842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.371269 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.371588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.371872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] PATH_INFO: `/v3/projects/b8edec2f81254cc7ad3fcbeb4e9c0315` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.373107 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8edec2f81254cc7ad3fcbeb4e9c0315)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.375982 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:46.404562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:46.434098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8edec2f81254cc7ad3fcbeb4e9c0315', 'request_id': 'req-8e7c5ed0-5fc8-47ff-ae20-209370174371'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:46.442027 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8edec2f81254cc7ad3fcbeb4e9c0315', 'request_id': 'req-8e7c5ed0-5fc8-47ff-ae20-209370174371'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:46.452902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8edec2f81254cc7ad3fcbeb4e9c0315', 'request_id': 'req-8e7c5ed0-5fc8-47ff-ae20-209370174371'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:46.471673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] The token cache is being invalidate because project b8edec2f81254cc7ad3fcbeb4e9c0315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:46.471960 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:46.472631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e7c5ed0-5fc8-47ff-ae20-209370174371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:46.474842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2769] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:46 2022] DELETE /identity/v3/projects/b8edec2f81254cc7ad3fcbeb4e9c0315 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:46.480692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.481151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.481403 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.501906 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:46.503210 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:46.607283 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1403/2770] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:46.618606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.621610 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.622697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.653611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:46.654891 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:46.663575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2771] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:29:46.669868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.681159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec2a62-37b1-4004-bc27-a72ca883a263 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.686027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.686679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.686972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.687250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.687982 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:46.688290 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.689897 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:46.704957 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1404/2772] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:29:46.709420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.721275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455e7e1-8d30-4a1a-9119-eabd4f7a3677 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.727113 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.727817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.728139 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.728487 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.729194 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.731158 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:46.762809 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2773] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:46 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:46.767047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.778190 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87602ecc-f108-41c6-bd04-da44859b3909 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.843223 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.844175 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.844477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.844832 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.845602 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.848189 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:46.874229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1405/2774] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:46 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:46.880823 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.892101 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cff182-aa58-4a69-9a4b-487667843d9a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.897477 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.898119 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.898406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.898694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.900135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.901769 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:46.914217 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2775] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:46.921146 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.933100 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ad12-07ae-4a60-b1cb-98d7125d134b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.939479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0610-5b0e-4f61-8d05-95669d73ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.940196 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0610-5b0e-4f61-8d05-95669d73ed25 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.940529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0610-5b0e-4f61-8d05-95669d73ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.940815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0610-5b0e-4f61-8d05-95669d73ed25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.941535 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0610-5b0e-4f61-8d05-95669d73ed25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.943169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0610-5b0e-4f61-8d05-95669d73ed25 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:46.955174 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1406/2776] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:46.959585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:46.971121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40560fe-65ce-4354-8039-3a336b0e0edd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:46.976089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:46.976777 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:46.977109 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:46.977377 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] PATH_INFO: `/v3/projects/9b589607fa3c402781a913338a7226a2/users/69df0806a8a0441a9d47d01c0753c8da/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:46.995607 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b589607fa3c402781a913338a7226a2, user_id=69df0806a8a0441a9d47d01c0753c8da, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:46.998367 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:47.015045 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2777] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:46 2022] PUT /identity/v3/projects/9b589607fa3c402781a913338a7226a2/users/69df0806a8a0441a9d47d01c0753c8da/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:47.026445 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:47.026743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:47.027024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:47.048023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:47.062308 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] MFA Rules not processed for user `69df0806a8a0441a9d47d01c0753c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:47.065621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:47.136334 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1407/2778] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:29:47.147313 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:47.149727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:47.218177 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625664b6-db6d-4d81-ac7f-a41339e7c25b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:47.222233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90995b6-367e-459c-ab12-209c571bb42a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:47.222881 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90995b6-367e-459c-ab12-209c571bb42a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:47.223159 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90995b6-367e-459c-ab12-209c571bb42a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:47.223483 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90995b6-367e-459c-ab12-209c571bb42a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:47.232822 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90995b6-367e-459c-ab12-209c571bb42a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:47.234311 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90995b6-367e-459c-ab12-209c571bb42a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:47.244543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2779] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:29:48.227099 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:48.229121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:48.298785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50f0d1-a2fe-4e18-9b6b-f69494dfa448 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:48.303514 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:48.304172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:48.304492 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:48.304771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] PATH_INFO: `/v3/users/633c0554f33842af9a9b9278dd21726d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:48.307163 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=633c0554f33842af9a9b9278dd21726d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:48.309460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:48.358056 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633c0554f33842af9a9b9278dd21726d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:48.365633 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633c0554f33842af9a9b9278dd21726d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:48.366263 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633c0554f33842af9a9b9278dd21726d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:48.366902 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633c0554f33842af9a9b9278dd21726d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:48.380227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633c0554f33842af9a9b9278dd21726d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:48.394352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1408/2780] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:48 2022] DELETE /identity/v3/users/633c0554f33842af9a9b9278dd21726d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:48.399145 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:48.399649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:48.399771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:48.419674 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:48.420883 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:48.467500 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:48.489271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2781] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:48.499943 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:48.501385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:48.522926 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:48.575487 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5f46-7e40-4961-b5ac-6fdb56fd72ac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:48.579164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:48.579820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:48.580105 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:48.580378 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:48.589639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:48.591071 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:48.603431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1409/2782] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:48.908741 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:48.910613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:48.925050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc299b-e402-4382-a19a-68009b2a4e02 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:48.931876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:48.932545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:48.932830 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:48.933191 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] PATH_INFO: `/v3/projects/8280e7a952d347c3b54d8a1d49896ca2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:48.934349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=8280e7a952d347c3b54d8a1d49896ca2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:48.936465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:48.996955 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8280e7a952d347c3b54d8a1d49896ca2', 'request_id': 'req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.010622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8280e7a952d347c3b54d8a1d49896ca2', 'request_id': 'req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.018964 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8280e7a952d347c3b54d8a1d49896ca2', 'request_id': 'req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.032378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] The token cache is being invalidate because project 8280e7a952d347c3b54d8a1d49896ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:49.032679 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.033332 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.034876 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2783] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:48 2022] DELETE /identity/v3/projects/8280e7a952d347c3b54d8a1d49896ca2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:49.038931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:49.061121 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efbe91-3cbf-4715-8863-d2ca6b6712ab service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:49.145802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:49.149505 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.150936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.151271 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] PATH_INFO: `/v3/users/206b0c60b13749aa9184f2bb8b7000d3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.152724 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=206b0c60b13749aa9184f2bb8b7000d3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:49.154701 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:49.203586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206b0c60b13749aa9184f2bb8b7000d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.211566 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206b0c60b13749aa9184f2bb8b7000d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.212218 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206b0c60b13749aa9184f2bb8b7000d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.212915 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206b0c60b13749aa9184f2bb8b7000d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.225491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51ec8be0-eb82-4f54-8996-311fa53ed7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206b0c60b13749aa9184f2bb8b7000d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.238393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1410/2784] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:49 2022] DELETE /identity/v3/users/206b0c60b13749aa9184f2bb8b7000d3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:49.525049 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:49.526538 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:49.541902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5917a-a8b9-4a8d-b51a-ac3b175de9ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:49.606529 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:49.607196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.607499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.607814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] PATH_INFO: `/v3/projects/d62eb8f356b0471caf9e792a9a8d99fe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.608990 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62eb8f356b0471caf9e792a9a8d99fe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:49.611037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:49.657727 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62eb8f356b0471caf9e792a9a8d99fe', 'request_id': 'req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.664309 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62eb8f356b0471caf9e792a9a8d99fe', 'request_id': 'req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.671218 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62eb8f356b0471caf9e792a9a8d99fe', 'request_id': 'req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.680556 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:49.685598 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] The token cache is being invalidate because project d62eb8f356b0471caf9e792a9a8d99fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:49.686121 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.687179 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4eb4bf22-3aae-4f6e-9ef2-a72adbea4e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:49.689118 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2785] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:49 2022] DELETE /identity/v3/projects/d62eb8f356b0471caf9e792a9a8d99fe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:49.694075 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.694370 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.694680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.716859 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:49.717805 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:49.751229 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:49.785141 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1411/2786] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:49.794327 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.794619 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.794904 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.822454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:49.823761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:49.833224 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2787] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:49.838592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:49.848460 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b97e3c-ee1b-4bbf-b37e-4948fc985f2e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:49.854173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:49.854931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.855258 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.855786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.856753 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:49.857114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:49.859090 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:49.871536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1412/2788] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:29:49.875874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:49.885658 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26d506-2287-4009-bf0d-517b2089e947 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:49.890421 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:49.891038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.891324 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.891632 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.892304 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:49.894220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:49.923801 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2789] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:49 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:49.928007 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:49.938085 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcbb5c-3529-4ab2-a0ba-d4f380b78136 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:49.995865 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:49.996491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:49.996808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:49.997097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:49.997770 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:49.999743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.024901 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1413/2790] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:49 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:50.031417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:50.042545 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0ad7c-6345-487a-a36b-7056c87fab18 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:50.047133 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:50.047807 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.048098 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.048378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.049069 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:50.050671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.061887 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2791] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:50.066749 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:50.078358 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ed908-d5c7-4bfc-b163-d54569da21d1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:50.083611 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b372fb-d704-4b96-b68f-4cc3b9543cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:50.084235 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b372fb-d704-4b96-b68f-4cc3b9543cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.084548 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b372fb-d704-4b96-b68f-4cc3b9543cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.084833 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b372fb-d704-4b96-b68f-4cc3b9543cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.085584 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b372fb-d704-4b96-b68f-4cc3b9543cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:50.088326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b372fb-d704-4b96-b68f-4cc3b9543cc4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.103563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1414/2792] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:50.104875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:50.115589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aba4c2-835e-4d7a-9705-e8dcb68dfade admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:50.120380 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:50.121068 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.121359 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.121644 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] PATH_INFO: `/v3/projects/649cc315561f444092f2941a717e9d1d/users/9f40813a052447babc6f9a102c74cadd/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.137853 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649cc315561f444092f2941a717e9d1d, user_id=9f40813a052447babc6f9a102c74cadd, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:50.140606 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.155604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2793] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:50 2022] PUT /identity/v3/projects/649cc315561f444092f2941a717e9d1d/users/9f40813a052447babc6f9a102c74cadd/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:50.168191 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.168607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.168992 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.199503 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] MFA Rules not processed for user `9f40813a052447babc6f9a102c74cadd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:50.200758 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:50.277526 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1415/2794] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:50.288725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:50.290781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:50.355144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f43c4-11a2-4c39-bfd9-bcf1201735f7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:50.359694 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871d755-5d3a-44f2-a8fd-35f94a22de2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:50.360652 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871d755-5d3a-44f2-a8fd-35f94a22de2c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.360958 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871d755-5d3a-44f2-a8fd-35f94a22de2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.361338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871d755-5d3a-44f2-a8fd-35f94a22de2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.371673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871d755-5d3a-44f2-a8fd-35f94a22de2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:50.373455 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871d755-5d3a-44f2-a8fd-35f94a22de2c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.383187 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2795] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:50.405125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:50.455738 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:50.485047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300a67c-6fd2-47f1-81f4-dc9458c7a8e1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:50.490084 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:50.490879 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.491383 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.491771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] PATH_INFO: `/v3/users/9f40813a052447babc6f9a102c74cadd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.495567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f40813a052447babc6f9a102c74cadd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:50.496137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.550147 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f40813a052447babc6f9a102c74cadd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:50.557886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f40813a052447babc6f9a102c74cadd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:50.558592 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f40813a052447babc6f9a102c74cadd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:50.559204 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f40813a052447babc6f9a102c74cadd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:50.573311 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f40813a052447babc6f9a102c74cadd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:50.585931 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1416/2796] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:50 2022] DELETE /identity/v3/users/9f40813a052447babc6f9a102c74cadd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:50.590654 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.591193 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.591294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.611294 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:50.612597 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:50.674828 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2797] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:50.686088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:50.687589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:50.746771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c2642-27ca-47f3-a9b2-1fcdb668facd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:50.750448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6b97-ee83-4b00-aab3-175b2ca249ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:50.751069 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6b97-ee83-4b00-aab3-175b2ca249ff service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:50.751385 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6b97-ee83-4b00-aab3-175b2ca249ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:50.751672 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6b97-ee83-4b00-aab3-175b2ca249ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:50.761540 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6b97-ee83-4b00-aab3-175b2ca249ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:50.763012 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6b97-ee83-4b00-aab3-175b2ca249ff service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:50.772852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1417/2798] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:51.054178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.056196 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:51.068847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08073507-6802-4442-9a3b-a15e8865dff7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.074437 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.075095 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.075404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.075690 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] PATH_INFO: `/v3/projects/649cc315561f444092f2941a717e9d1d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.076878 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=649cc315561f444092f2941a717e9d1d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.078814 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.118816 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:51.130061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649cc315561f444092f2941a717e9d1d', 'request_id': 'req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:51.136262 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649cc315561f444092f2941a717e9d1d', 'request_id': 'req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:51.142493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649cc315561f444092f2941a717e9d1d', 'request_id': 'req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:51.157649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] The token cache is being invalidate because project 649cc315561f444092f2941a717e9d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:51.157946 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:51.158626 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dd9dac2-789a-4b0c-8dd8-9353a9ffd4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:51.160195 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2799] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:51 2022] DELETE /identity/v3/projects/649cc315561f444092f2941a717e9d1d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:51.166513 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.166810 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.167094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.186721 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:51.187894 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:51.239172 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:51.251040 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1418/2800] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:51.259465 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.259766 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.260050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.280115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:51.281338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:51.289366 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2801] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 05:29:51.293649 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.303667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c378f-29e8-4640-b7f0-02b91de30173 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.308773 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.309380 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.309662 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.309972 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.310691 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:51.311000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.312637 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.324979 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1419/2802] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:29:51.329336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.339393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b7473-86dd-4dba-81e5-4cf9e2d9dee0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.344356 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.344974 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.345256 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.345530 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.346203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.348172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.376666 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2803] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:51 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:51.380923 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.391042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f95a34-eb94-46c7-8342-2d267acc0b67 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.452665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.453156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.453462 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.453813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.454479 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.456608 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.480050 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1420/2804] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:51 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:51.484558 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.497431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75ec3a-a2c0-41e9-ad23-f5f561dd448c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.503064 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.503763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.504103 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.504415 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.505239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.507310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.521501 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2805] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:51.526911 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.543211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82affd05-86b9-438e-9ae0-9e923e76e015 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.548328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d0f20-0a66-4f43-bf84-302f1d1c0879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.548963 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d0f20-0a66-4f43-bf84-302f1d1c0879 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.549251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d0f20-0a66-4f43-bf84-302f1d1c0879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.549669 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d0f20-0a66-4f43-bf84-302f1d1c0879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.550239 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d0f20-0a66-4f43-bf84-302f1d1c0879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.552047 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d0f20-0a66-4f43-bf84-302f1d1c0879 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.566733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1421/2806] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:29:51.568398 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.580617 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ff96-97e8-434a-b24b-0128e0cb366a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.586074 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.587752 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.588041 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.588506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] PATH_INFO: `/v3/projects/e886cdbe0a894de494b92b2d1db72e05/users/898c72f0b44e487d9cb0a3ae4bd0e32d/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.605078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e886cdbe0a894de494b92b2d1db72e05, user_id=898c72f0b44e487d9cb0a3ae4bd0e32d, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.608146 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.635078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2807] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:51 2022] PUT /identity/v3/projects/e886cdbe0a894de494b92b2d1db72e05/users/898c72f0b44e487d9cb0a3ae4bd0e32d/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:51.637405 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.637778 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.638068 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.671615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] MFA Rules not processed for user `898c72f0b44e487d9cb0a3ae4bd0e32d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:51.672940 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:51.752250 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1422/2808] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:51.763765 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:51.765834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:51.835322 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc719-afd5-4ed6-9039-3c8fddaee30d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:51.840368 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c75d78-705b-400d-8e6a-b841bcd5909a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:51.841005 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75d78-705b-400d-8e6a-b841bcd5909a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:51.841293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75d78-705b-400d-8e6a-b841bcd5909a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:51.841575 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75d78-705b-400d-8e6a-b841bcd5909a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:51.850784 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c75d78-705b-400d-8e6a-b841bcd5909a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:51.853709 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c75d78-705b-400d-8e6a-b841bcd5909a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:51.867246 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2809] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:54.460806 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:54.526343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:54.539037 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4a8ae-bb4b-4fff-a0d7-c2c3ce7648c0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:54.543605 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:54.544265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:54.544543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:54.544929 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] PATH_INFO: `/v3/users/898c72f0b44e487d9cb0a3ae4bd0e32d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:54.546003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=898c72f0b44e487d9cb0a3ae4bd0e32d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:54.547991 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:54.604024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '898c72f0b44e487d9cb0a3ae4bd0e32d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:54.612441 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '898c72f0b44e487d9cb0a3ae4bd0e32d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:54.613259 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '898c72f0b44e487d9cb0a3ae4bd0e32d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:54.613939 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '898c72f0b44e487d9cb0a3ae4bd0e32d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:54.627673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '898c72f0b44e487d9cb0a3ae4bd0e32d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:54.641821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1423/2810] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:54 2022] DELETE /identity/v3/users/898c72f0b44e487d9cb0a3ae4bd0e32d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:54.647036 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:54.647580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:54.647796 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:54.669763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:54.671687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:54.745584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2811] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:54.760524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:54.762734 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:54.828313 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ff2d8-2d24-446b-bea1-7cdb6f88d2d3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:54.832657 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e496d7b-f005-4895-9db9-69105c018a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:54.833287 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e496d7b-f005-4895-9db9-69105c018a73 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:54.833574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e496d7b-f005-4895-9db9-69105c018a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:54.833857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e496d7b-f005-4895-9db9-69105c018a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:54.843775 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e496d7b-f005-4895-9db9-69105c018a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:54.845224 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e496d7b-f005-4895-9db9-69105c018a73 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:54.855201 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1424/2812] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:55.134163 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.135983 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:55.143604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:55.168497 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3441-9e68-49bf-8483-c46334e994f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.173203 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.174002 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.174288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.174567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] PATH_INFO: `/v3/projects/e886cdbe0a894de494b92b2d1db72e05` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.175999 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e886cdbe0a894de494b92b2d1db72e05)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.177905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.237213 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e886cdbe0a894de494b92b2d1db72e05', 'request_id': 'req-be501877-0c96-4f99-8d89-69b20084cff0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:55.245172 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e886cdbe0a894de494b92b2d1db72e05', 'request_id': 'req-be501877-0c96-4f99-8d89-69b20084cff0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:55.251264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e886cdbe0a894de494b92b2d1db72e05', 'request_id': 'req-be501877-0c96-4f99-8d89-69b20084cff0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:55.265016 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] The token cache is being invalidate because project e886cdbe0a894de494b92b2d1db72e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:55.265314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:55.265932 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be501877-0c96-4f99-8d89-69b20084cff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:55.267588 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2813] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:55 2022] DELETE /identity/v3/projects/e886cdbe0a894de494b92b2d1db72e05 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:55.272750 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.273049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.273428 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.294024 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:55.295245 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:55.356219 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1425/2814] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:55.365276 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.365571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.365871 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.386531 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:55.387805 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:55.396394 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2815] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:55.401154 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.411411 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012c95-1c44-4002-a5b8-2f4251858c6a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.416730 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.417343 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.417626 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.417904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.418599 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:29:55.418907 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.420546 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.435993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1426/2816] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:29:55.439182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.450175 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710d40f-d3db-47f2-b003-af2d18386124 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.455485 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.456089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.456450 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.456795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.457715 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.459755 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.490046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2817] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:29:55 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:29:55.495646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.500715 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:55.506891 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da664e13-74bb-4581-9758-e3f4d04723db admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.568409 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.569030 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.569333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.569587 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.570288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.572373 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.594157 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1427/2818] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:29:55 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:29:55.598100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.609544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb234-2bd6-4c72-8b21-e05385d70b64 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.614519 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.615147 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.615484 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.615770 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.616471 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.618144 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.628539 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2819] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:55.632746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.643215 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972929a5-6973-4d42-8726-915388d0c2e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.648240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd1deb-e3f0-4b4b-b8ed-2d14ecdcdf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.649384 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd1deb-e3f0-4b4b-b8ed-2d14ecdcdf01 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.649673 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd1deb-e3f0-4b4b-b8ed-2d14ecdcdf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.649948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd1deb-e3f0-4b4b-b8ed-2d14ecdcdf01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.650639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd1deb-e3f0-4b4b-b8ed-2d14ecdcdf01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.652284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd1deb-e3f0-4b4b-b8ed-2d14ecdcdf01 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.662948 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1428/2820] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:29:55.667961 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.678950 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0302e-96c6-433e-8e5b-718d7fdd4c5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.684693 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.685391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.685675 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.685953 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] PATH_INFO: `/v3/projects/0fb4ada21ebe4a4a9cb9d2046b861e7e/users/044f8045bb1344b190edc32bf9c2e53b/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.705553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb4ada21ebe4a4a9cb9d2046b861e7e, user_id=044f8045bb1344b190edc32bf9c2e53b, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.708300 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.723378 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2821] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:29:55 2022] PUT /identity/v3/projects/0fb4ada21ebe4a4a9cb9d2046b861e7e/users/044f8045bb1344b190edc32bf9c2e53b/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:55.734359 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.734659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.734941 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.762932 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] MFA Rules not processed for user `044f8045bb1344b190edc32bf9c2e53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:55.764202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:55.845665 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1429/2822] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:55.857659 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:55.859543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:55.929736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a52cb-9e47-488f-b38b-d4b9583c47cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:55.934670 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f19d52-67fc-4ffe-84f5-ba762701054d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:55.935293 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f19d52-67fc-4ffe-84f5-ba762701054d service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:55.935631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f19d52-67fc-4ffe-84f5-ba762701054d service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:55.935914 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f19d52-67fc-4ffe-84f5-ba762701054d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:55.945896 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f19d52-67fc-4ffe-84f5-ba762701054d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:55.947743 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f19d52-67fc-4ffe-84f5-ba762701054d service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:55.958481 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2823] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:57.465935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:57.467434 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:57.550265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef676041-8926-439e-b702-ea490b0cf834 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:57.551525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:57.552464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:57.552808 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:57.553088 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] PATH_INFO: `/v3/users/69df0806a8a0441a9d47d01c0753c8da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:57.554443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=69df0806a8a0441a9d47d01c0753c8da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:57.556917 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:57.571725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:57.607785 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69df0806a8a0441a9d47d01c0753c8da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:57.615211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69df0806a8a0441a9d47d01c0753c8da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:57.615890 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69df0806a8a0441a9d47d01c0753c8da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:57.616504 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69df0806a8a0441a9d47d01c0753c8da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:57.629610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69df0806a8a0441a9d47d01c0753c8da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:57.643137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1430/2824] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:29:57 2022] DELETE /identity/v3/users/69df0806a8a0441a9d47d01c0753c8da => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:29:57.648050 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:57.649279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:57.649279 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:57.668404 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:29:57.669610 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:57.688920 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:29:57.731660 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2825] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:29:57.741537 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:57.742942 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:57.801389 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d32241-be93-4a34-aaa7-cde8af52b2be admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:57.805628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:57.806251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:57.806538 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:57.806820 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:57.816138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:57.817582 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:57.828345 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1431/2826] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:29:58.094230 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:29:58.095897 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:29:58.108076 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5a2d4-e818-44ea-8cfc-22ec17df1fdb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:29:58.112725 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:29:58.113370 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:29:58.113653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:29:58.113987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] PATH_INFO: `/v3/projects/9b589607fa3c402781a913338a7226a2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:29:58.115321 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b589607fa3c402781a913338a7226a2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:29:58.117840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:29:58.166326 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b589607fa3c402781a913338a7226a2', 'request_id': 'req-e115d996-8bc0-47cf-ac44-0d6f60ada72f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:58.173221 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b589607fa3c402781a913338a7226a2', 'request_id': 'req-e115d996-8bc0-47cf-ac44-0d6f60ada72f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:58.178984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b589607fa3c402781a913338a7226a2', 'request_id': 'req-e115d996-8bc0-47cf-ac44-0d6f60ada72f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:58.193011 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] The token cache is being invalidate because project 9b589607fa3c402781a913338a7226a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:29:58.193375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:58.194048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e115d996-8bc0-47cf-ac44-0d6f60ada72f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:29:58.196154 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2827] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:29:58 2022] DELETE /identity/v3/projects/9b589607fa3c402781a913338a7226a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:01.483242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:01.484737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:01.499675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1568d9-b2dc-472b-9e4f-198f67c64f93 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:01.560377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:01.561031 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:01.561322 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:01.561607 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] PATH_INFO: `/v3/users/a5546058044b433d9907bea616cc9aa8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:01.562838 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5546058044b433d9907bea616cc9aa8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:01.564813 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:01.601151 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:01.611868 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5546058044b433d9907bea616cc9aa8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:01.620170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5546058044b433d9907bea616cc9aa8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:01.620924 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5546058044b433d9907bea616cc9aa8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:01.621594 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5546058044b433d9907bea616cc9aa8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:01.634094 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5546058044b433d9907bea616cc9aa8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:01.646659 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1432/2828] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:30:01 2022] DELETE /identity/v3/users/a5546058044b433d9907bea616cc9aa8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:01.651754 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:01.652242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:01.652361 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:01.670942 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:01.672184 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:01.730943 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2829] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:01.740497 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:01.741889 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:01.800781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a1cf3-a579-4a5f-9471-583a16737764 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:01.804507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e8415-9e3e-424d-8a5a-05aba5c9516d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:01.805712 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e8415-9e3e-424d-8a5a-05aba5c9516d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:01.806041 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e8415-9e3e-424d-8a5a-05aba5c9516d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:01.806265 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e8415-9e3e-424d-8a5a-05aba5c9516d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:01.815598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e8415-9e3e-424d-8a5a-05aba5c9516d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:01.817027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e8415-9e3e-424d-8a5a-05aba5c9516d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:01.827387 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1433/2830] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:30:02.107842 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.109336 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:02.122260 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd2c57-e201-4fbf-9349-39a96f4c1bf8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.126980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.127668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.127987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.128268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] PATH_INFO: `/v3/projects/409bdfebeee241389844e3015aab0a86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.129442 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=409bdfebeee241389844e3015aab0a86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.131379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.179199 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:02.183444 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409bdfebeee241389844e3015aab0a86', 'request_id': 'req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:02.190761 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409bdfebeee241389844e3015aab0a86', 'request_id': 'req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:02.197288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409bdfebeee241389844e3015aab0a86', 'request_id': 'req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:02.211829 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] The token cache is being invalidate because project 409bdfebeee241389844e3015aab0a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:30:02.212124 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:02.212771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3cc05dd7-31c8-4009-8c5c-fb488a02bda3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:02.214273 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2831] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:30:02 2022] DELETE /identity/v3/projects/409bdfebeee241389844e3015aab0a86 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:02.219275 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.219589 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.219864 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.241524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:02.242910 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:02.302353 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1434/2832] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:02.312026 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.312323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.312633 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.332834 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:02.334061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:02.342751 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2833] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:02.347113 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.357761 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc1f29-8a9d-42f8-adba-72fe68ed9fa9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.362375 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.362990 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.363277 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.363595 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.364288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:30:02.364600 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.366211 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.378206 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1435/2834] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 05:30:02.382720 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.392120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c545ad-8dbd-4a2f-ba2e-5d6df8737aa7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.397169 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.397780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.398066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.398350 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.399593 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.401513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.429672 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2835] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:30:02 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:30:02.434001 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.446114 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3f59-5f45-4f09-bcc9-6d42b9657c25 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.486656 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:02.505137 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.505695 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.506003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.506285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.506968 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.509049 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.531842 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1436/2836] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:30:02 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:30:02.536189 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.547089 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f200-fccc-47c2-8092-18096a5989d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.551776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.552342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.552624 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.552905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.553611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.555379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.565423 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2837] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:30:02.569683 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.579863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2fb96-ce8d-4ac0-9f3f-4bf9edf57cc5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.584518 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bcc84-22a8-47b7-917c-2480c6221fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.585129 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bcc84-22a8-47b7-917c-2480c6221fea admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.585417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bcc84-22a8-47b7-917c-2480c6221fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.585699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bcc84-22a8-47b7-917c-2480c6221fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.586983 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bcc84-22a8-47b7-917c-2480c6221fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.588628 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bcc84-22a8-47b7-917c-2480c6221fea admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.599164 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1437/2838] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:30:02.604013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.613865 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e6038-49ca-47c6-a2e7-e1c123d12332 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.618512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.619192 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.619571 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.619850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] PATH_INFO: `/v3/projects/294d13c1380943e4ab3516e99a3954bc/users/745938576f23423d844c0b7fd17d5de6/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.634949 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=294d13c1380943e4ab3516e99a3954bc, user_id=745938576f23423d844c0b7fd17d5de6, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.637763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.652236 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2839] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:30:02 2022] PUT /identity/v3/projects/294d13c1380943e4ab3516e99a3954bc/users/745938576f23423d844c0b7fd17d5de6/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:02.662572 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94268-2338-4373-a33d-bf00607af010 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.662871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94268-2338-4373-a33d-bf00607af010 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.663159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94268-2338-4373-a33d-bf00607af010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.692253 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42b94268-2338-4373-a33d-bf00607af010 None None] MFA Rules not processed for user `745938576f23423d844c0b7fd17d5de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:02.693505 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42b94268-2338-4373-a33d-bf00607af010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:02.758815 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1438/2840] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:02.770073 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:02.772066 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:02.830987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359ee77-56ab-46c2-a409-20e2259aff6c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:02.834634 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d2c2c-de47-4029-9d95-c86adc55b4ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a47f6ceb2a146b092ebeb72e8679335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:02.835254 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d2c2c-de47-4029-9d95-c86adc55b4ab service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:02.835567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d2c2c-de47-4029-9d95-c86adc55b4ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:02.835844 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d2c2c-de47-4029-9d95-c86adc55b4ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:02.845059 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d2c2c-de47-4029-9d95-c86adc55b4ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:02.846499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d2c2c-de47-4029-9d95-c86adc55b4ab service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:02.855574 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2841] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:30:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:30:11.965058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94268-2338-4373-a33d-bf00607af010 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:12.035821 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94268-2338-4373-a33d-bf00607af010 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:12.040709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:12.042063 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:12.042357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:12.042639 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] PATH_INFO: `/v3/users/745938576f23423d844c0b7fd17d5de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:12.043987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] RBAC: Authorizing `identity:delete_user(user_id=745938576f23423d844c0b7fd17d5de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:12.049261 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:12.096148 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745938576f23423d844c0b7fd17d5de6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.100310 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:12.105045 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745938576f23423d844c0b7fd17d5de6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.106205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745938576f23423d844c0b7fd17d5de6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.107233 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745938576f23423d844c0b7fd17d5de6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.121559 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745938576f23423d844c0b7fd17d5de6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.137667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1439/2842] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:30:11 2022] DELETE /identity/v3/users/745938576f23423d844c0b7fd17d5de6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:12.142601 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:12.143160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:12.143160 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:12.164278 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:12.165488 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:12.233454 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2843] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:12.243857 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:12.245646 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:12.303185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b4c9d-f013-4b78-b2a9-f764d297510e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:12.307302 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:12.307960 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:12.308251 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:12.308532 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:12.317723 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:12.319169 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:12.328783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1440/2844] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:30:12.601013 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:12.603046 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:12.620443 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6a011-2d84-435e-a815-9d73b5bbc17e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:12.626070 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:12.626691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:12.626973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:12.627263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] PATH_INFO: `/v3/projects/294d13c1380943e4ab3516e99a3954bc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:12.628439 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=294d13c1380943e4ab3516e99a3954bc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:12.630353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:12.641429 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:12.685211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294d13c1380943e4ab3516e99a3954bc', 'request_id': 'req-71295973-ffc3-493b-bcfa-0c0d3af37145'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.691308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294d13c1380943e4ab3516e99a3954bc', 'request_id': 'req-71295973-ffc3-493b-bcfa-0c0d3af37145'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.697956 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294d13c1380943e4ab3516e99a3954bc', 'request_id': 'req-71295973-ffc3-493b-bcfa-0c0d3af37145'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.710577 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] The token cache is being invalidate because project 294d13c1380943e4ab3516e99a3954bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:30:12.710872 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.711553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71295973-ffc3-493b-bcfa-0c0d3af37145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:12.713649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2845] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:30:12 2022] DELETE /identity/v3/projects/294d13c1380943e4ab3516e99a3954bc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:16.410317 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:16.411801 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:16.424962 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b8516-9df7-4ce2-839c-50c78bc81909 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:16.484525 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:16.485079 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:16.485364 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:16.485638 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] PATH_INFO: `/v3/users/044f8045bb1344b190edc32bf9c2e53b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:16.486800 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044f8045bb1344b190edc32bf9c2e53b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:16.488746 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:16.537739 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044f8045bb1344b190edc32bf9c2e53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:16.545666 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044f8045bb1344b190edc32bf9c2e53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:16.546427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044f8045bb1344b190edc32bf9c2e53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:16.547046 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044f8045bb1344b190edc32bf9c2e53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:16.560351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044f8045bb1344b190edc32bf9c2e53b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:16.562377 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:16.576786 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1441/2846] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:30:16 2022] DELETE /identity/v3/users/044f8045bb1344b190edc32bf9c2e53b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:16.581314 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:16.581824 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:16.581947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:16.602691 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:16.604584 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:16.666986 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2847] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:16.676768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:16.678168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:16.738880 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab13a6a-e3b0-4a04-ac25-cb8f07330232 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:16.742536 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e37e7b-9a0e-4dec-bcfb-e0d5d8a67377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:16.743159 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e37e7b-9a0e-4dec-bcfb-e0d5d8a67377 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:16.743486 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e37e7b-9a0e-4dec-bcfb-e0d5d8a67377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:16.743754 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e37e7b-9a0e-4dec-bcfb-e0d5d8a67377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:16.753771 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e37e7b-9a0e-4dec-bcfb-e0d5d8a67377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:16.755222 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e37e7b-9a0e-4dec-bcfb-e0d5d8a67377 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:16.765227 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1442/2848] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:30:17.048393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.049870 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:17.063622 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523234a0-a1f7-4a4d-aa95-581a35ffb132 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.069120 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.069820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.070239 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.070516 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] PATH_INFO: `/v3/projects/0fb4ada21ebe4a4a9cb9d2046b861e7e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.072037 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb4ada21ebe4a4a9cb9d2046b861e7e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.074082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.129544 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb4ada21ebe4a4a9cb9d2046b861e7e', 'request_id': 'req-b96d0423-1824-4041-9db3-3a11e938ab03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:17.136222 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb4ada21ebe4a4a9cb9d2046b861e7e', 'request_id': 'req-b96d0423-1824-4041-9db3-3a11e938ab03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:17.142417 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb4ada21ebe4a4a9cb9d2046b861e7e', 'request_id': 'req-b96d0423-1824-4041-9db3-3a11e938ab03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:17.157680 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] The token cache is being invalidate because project 0fb4ada21ebe4a4a9cb9d2046b861e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:30:17.157981 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:17.158637 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b96d0423-1824-4041-9db3-3a11e938ab03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:30:17.160416 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2849] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:30:17 2022] DELETE /identity/v3/projects/0fb4ada21ebe4a4a9cb9d2046b861e7e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:17.165392 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.165684 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.166477 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.192896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:17.194134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:17.261015 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1443/2850] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:17.328589 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.329126 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.329419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.350984 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:17.352263 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:17.363918 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2851] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:17.366848 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.377185 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8ee87-9e06-4e22-8ff9-a235aa3cd90f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.381877 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.382507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.382793 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.383076 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.383850 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 05:30:17.384166 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.386134 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.398839 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1444/2852] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 05:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 05:30:17.403778 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.413122 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f787b1-e17e-4e33-bfea-331f29fa8e29 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.418757 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.419406 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.419697 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.419980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.420650 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.422692 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.432603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:17.456987 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2853] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:30:17 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:30:17.458733 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.471682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ee062-92e7-4ee7-bfcb-a4e0c5b0100e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.534475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.535111 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.535424 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.535710 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.536406 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.538360 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.565934 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1445/2854] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:30:17 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 05:30:17.567167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.581265 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945583-be77-4807-ace1-78d69207b0e6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.586014 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.586641 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.586927 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.587200 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.588687 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.590793 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.604744 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2855] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 05:30:17.606033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.616729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20c248-d986-4e0a-9049-94bc7a2b4c48 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.621955 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e7772-d8f1-4c63-bceb-2cb4ba794e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.622598 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e7772-d8f1-4c63-bceb-2cb4ba794e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.622886 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e7772-d8f1-4c63-bceb-2cb4ba794e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.623167 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e7772-d8f1-4c63-bceb-2cb4ba794e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.623918 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e7772-d8f1-4c63-bceb-2cb4ba794e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.625576 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e7772-d8f1-4c63-bceb-2cb4ba794e64 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.631692 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:17.640240 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1446/2856] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:30:17.644344 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.656048 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de61e04-2be0-41a1-8dac-a8f90c317ab6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.661388 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.662086 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.662373 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.662653 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] PATH_INFO: `/v3/projects/3d61aa3cfb59443d849c3d09d839cff5/users/984ce8ca6aac41eab2c998c17d3820c5/roles/f5c39945c9694134955532988101506f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.679072 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d61aa3cfb59443d849c3d09d839cff5, user_id=984ce8ca6aac41eab2c998c17d3820c5, role_id=f5c39945c9694134955532988101506f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.681905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.696780 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2857] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:30:17 2022] PUT /identity/v3/projects/3d61aa3cfb59443d849c3d09d839cff5/users/984ce8ca6aac41eab2c998c17d3820c5/roles/f5c39945c9694134955532988101506f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:17.702621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2d9c-f9e3-4bb8-9074-6ff4eb96e6d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.703152 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2d9c-f9e3-4bb8-9074-6ff4eb96e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.703515 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2d9c-f9e3-4bb8-9074-6ff4eb96e6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.724524 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-24db2d9c-f9e3-4bb8-9074-6ff4eb96e6d5 None None] MFA Rules not processed for user `a74b2847a9d0472682c985005f47d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:17.725768 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24db2d9c-f9e3-4bb8-9074-6ff4eb96e6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:17.793618 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1447/2858] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:17.800393 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:17.802271 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:17.866408 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703075ce-cafa-425f-a92e-f9e6127e1895 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:17.870862 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:17.871613 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:17.871900 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:17.872248 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:17.882451 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:17.883973 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:17.895000 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2859] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:30:22.063959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.064475 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.064717 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.108187 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] MFA Rules not processed for user `984ce8ca6aac41eab2c998c17d3820c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:22.109449 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:22.230034 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1448/2860] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:22.234663 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:22.250630 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac5966-5b16-4681-9563-779389bc0961 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:22.256061 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:22.256646 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.256933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.257215 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.257886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:22.260407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:22.291736 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2861] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 05:30:22 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:30:22.295791 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:22.307676 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb047f-d07b-4f5c-b0bd-30522be4a818 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:22.376448 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:22.377097 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.377417 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.377699 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.378393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:22.380427 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:22.405491 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1449/2862] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 05:30:22 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 05:30:22.409585 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:22.422875 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7df630-adc0-46b5-b714-cacbc7b3bc89 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:22.436905 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:22.437532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.437980 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.438082 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.438947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:22.440749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:22.458773 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2863] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:30:22.463266 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:22.475596 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e276d9c-2ac2-4d2e-8876-bb51db7e823b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:22.482630 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:22.483399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.483804 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.484003 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] PATH_INFO: `/v3/projects/9bf85229f25c4bd7b45355082e2fc952/users/89202d24880044529b5fc5560b1b184c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.502433 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf85229f25c4bd7b45355082e2fc952, user_id=89202d24880044529b5fc5560b1b184c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:22.505173 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:22.508935 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:22.529621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1450/2864] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:30:22 2022] PUT /identity/v3/projects/9bf85229f25c4bd7b45355082e2fc952/users/89202d24880044529b5fc5560b1b184c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:22.533513 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:22.546308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada755-72a4-48ff-863f-5391f9c9bc26 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:22.630993 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afef504-5fcc-4719-849a-acc41db47d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:22.631682 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afef504-5fcc-4719-849a-acc41db47d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.631970 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afef504-5fcc-4719-849a-acc41db47d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.632235 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afef504-5fcc-4719-849a-acc41db47d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.632928 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afef504-5fcc-4719-849a-acc41db47d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:22.634562 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afef504-5fcc-4719-849a-acc41db47d76 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:22.647553 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2865] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 05:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 05:30:22.653004 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:22.665574 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aacfa-d694-4143-9a70-0267c85bd49f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:22.672571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:22.673351 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:22.673719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:22.674074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] PATH_INFO: `/v3/projects/9bf85229f25c4bd7b45355082e2fc952/users/89202d24880044529b5fc5560b1b184c/roles/8b45fb0997304e788e2bb021f59cc269` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:22.676465 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf85229f25c4bd7b45355082e2fc952, user_id=89202d24880044529b5fc5560b1b184c, role_id=8b45fb0997304e788e2bb021f59cc269)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:22.679655 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:22.693397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1451/2866] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 05:30:22 2022] PUT /identity/v3/projects/9bf85229f25c4bd7b45355082e2fc952/users/89202d24880044529b5fc5560b1b184c/roles/8b45fb0997304e788e2bb021f59cc269 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:30:28.198995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef6cf-3003-402c-b628-334fa3f2bca5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:28.199604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef6cf-3003-402c-b628-334fa3f2bca5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:28.199604 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef6cf-3003-402c-b628-334fa3f2bca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:28.240080 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e12ef6cf-3003-402c-b628-334fa3f2bca5 None None] MFA Rules not processed for user `89202d24880044529b5fc5560b1b184c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:28.241493 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e12ef6cf-3003-402c-b628-334fa3f2bca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:28.253167 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:28.372605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2867] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:28.396998 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:30:28.398959 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:28.545591 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a940b-824f-4d23-8dbb-e8a530ace3c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:30:28.551741 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:30:28.552393 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:28.552700 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:28.552976 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:28.626177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:30:28.627896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:30:28.638329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1452/2868] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:30:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:30:38.361456 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:38.633463 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:30:44.383542 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:30:44.384108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:30:44.384108 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:30:44.406667 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] MFA Rules not processed for user `984ce8ca6aac41eab2c998c17d3820c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:30:44.408277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:30:44.415420 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2869] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:30:54.399595 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:31:02.042577 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:02.099824 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f94b6-f2a5-4cac-b77d-acb1048e9323 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:02.103516 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:02.104122 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:02.104397 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:02.104680 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:02.106455 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:02.121543 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:02.124305 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:02.135399 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1453/2870] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:02.282582 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:02.284042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:02.296886 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232706e9-05da-43a8-9d15-1b363d873f2b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:02.300478 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:02.301092 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:02.301379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:02.301657 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:02.310714 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:02.312225 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:02.321728 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2871] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:12.130447 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=26 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:31:12.317469 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:31:32.591668 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:32.603081 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d455-8c19-47cf-a858-0007f12d4f03 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:32.606944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:32.607621 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:32.607904 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:32.608177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:32.610044 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:32.626335 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:32.628682 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bc9b2-bd94-4c3a-88f4-c796fe60e796 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:32.641789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1454/2872] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:42.635642 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:31:42.722713 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:42.731933 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1282fc-c457-4b2c-b533-d255d16b99a9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:42.735781 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:42.736427 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:42.736712 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:42.737008 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:42.738771 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:42.749182 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:42.750635 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b4cf1-0bfe-44fe-a9ca-d145febbea3a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:42.759910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2873] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:45.893309 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa615d08-133f-4d13-a756-fa1c1b04b743 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:45.893896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa615d08-133f-4d13-a756-fa1c1b04b743 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:45.893896 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa615d08-133f-4d13-a756-fa1c1b04b743 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:45.895205 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1455/2874] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:31:45.902178 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729a97d-413b-4aa7-ae8b-d0f58ebc9770 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:45.902673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729a97d-413b-4aa7-ae8b-d0f58ebc9770 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:45.902874 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729a97d-413b-4aa7-ae8b-d0f58ebc9770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:45.942419 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5729a97d-413b-4aa7-ae8b-d0f58ebc9770 None None] MFA Rules not processed for user `eac8ab3df29343c5bfa8928000f99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:31:45.944349 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5729a97d-413b-4aa7-ae8b-d0f58ebc9770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:45.951611 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2875] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:31:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:31:45.964278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa615d08-133f-4d13-a756-fa1c1b04b743 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:45.976431 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa615d08-133f-4d13-a756-fa1c1b04b743 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:45.979928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f331a3f-c6e4-4f20-8a71-66a0d2997f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:45.980579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f331a3f-c6e4-4f20-8a71-66a0d2997f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:45.980863 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f331a3f-c6e4-4f20-8a71-66a0d2997f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:45.981139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f331a3f-c6e4-4f20-8a71-66a0d2997f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:45.988622 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f331a3f-c6e4-4f20-8a71-66a0d2997f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:45.990170 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f331a3f-c6e4-4f20-8a71-66a0d2997f09 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:45.998737 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1456/2876] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:46.095629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b332-95dd-4074-8cb5-2b6bbbd3a164 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:46.095924 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b332-95dd-4074-8cb5-2b6bbbd3a164 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:46.096206 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b332-95dd-4074-8cb5-2b6bbbd3a164 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:46.097564 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2877] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 05:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 05:31:46.104033 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:46.104333 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:46.104767 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:46.143022 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] MFA Rules not processed for user `1d28b235f4434aa086cdfc0fc81c7551`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:31:46.144292 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:46.154425 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1457/2878] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 05:31:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:31:46.165894 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9b332-95dd-4074-8cb5-2b6bbbd3a164 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:46.178226 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9b332-95dd-4074-8cb5-2b6bbbd3a164 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:46.181850 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:46.182532 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:46.182820 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:46.183132 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:46.194325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:46.195852 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:46.207308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2879] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:46.269493 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:31:46.271138 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:46.340615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45894b-6d32-4760-be17-beb056e80e9f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:31:46.345679 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:31:46.346326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:31:46.346610 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:31:46.346887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:31:46.349131 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:31:46.360817 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:31:46.362352 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2622b1-7dab-46d6-9f54-ac660e6d7b76 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:31:46.372961 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1458/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:31:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:31:56.202223 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=21 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:31:56.368757 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=47 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:33:01.756310 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:01.768379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9270d0c-23f0-4f23-8946-c442c409ee76 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:01.774042 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:01.774673 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:01.775083 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:01.775264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:01.777400 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:01.791499 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:01.793268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:01.805977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2881] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:33:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:33:02.980027 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:02.980520 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:02.980625 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:03.000171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] MFA Rules not processed for user `984ce8ca6aac41eab2c998c17d3820c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:33:03.001432 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:03.008337 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1459/2882] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:33:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:33:03.329567 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:03.338885 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1298-88db-43a1-9eb7-bbc2df24453d service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:03.342636 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:03.343252 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:03.343583 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:03.343866 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:03.345668 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:03.355211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:03.356671 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:03.364749 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2883] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:33:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:33:10.288743 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:10.290288 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:10.318042 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef69fef-4854-4c8a-b054-f2886fa3205c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:10.323930 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:10.324567 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:10.324855 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:10.325139 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:10.326989 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:10.342563 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:10.343284 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:10.356344 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1460/2884] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:33:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:33:13.361156 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=12 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:33:15.383277 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:15.392342 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff17cc-5481-4c8f-bc1a-72ba433254ed service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:15.396639 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:15.397295 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:15.397599 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:15.397859 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:15.399776 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:15.469288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:15.470931 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:15.482997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2885] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:33:16.427312 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:16.451070 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239af6d-8748-4d0e-83db-f89ef3ab390d service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:16.458569 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d28b235f4434aa086cdfc0fc81c7551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:16.459452 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:16.459888 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:16.460329 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:16.463000 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:16.482349 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:16.485237 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:16.505756 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1461/2886] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:33:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:33:23.292492 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:23.295391 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:23.312540 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d4e0b-9a62-428d-8dc3-6e7084410a31 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:23.319249 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:23.320353 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:23.320833 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:23.321211 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] PATH_INFO: `/v3/users/5b1d0da1157a46c283bd442ad14d31b5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:23.322902 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1d0da1157a46c283bd442ad14d31b5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:23.325308 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:23.392402 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1d0da1157a46c283bd442ad14d31b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.393290 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1d0da1157a46c283bd442ad14d31b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.403091 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1d0da1157a46c283bd442ad14d31b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.403926 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1d0da1157a46c283bd442ad14d31b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.419923 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1d0da1157a46c283bd442ad14d31b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.436325 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2887] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:33:23 2022] DELETE /identity/v3/users/5b1d0da1157a46c283bd442ad14d31b5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:33:23.709729 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:23.730112 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112d6d2-4847-4088-907e-d7c7ee16265c service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:23.812560 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:23.813241 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:23.813529 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:23.813814 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] PATH_INFO: `/v3/projects/719297a86f0a4930b4daf3b9f663e472` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:23.815155 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=719297a86f0a4930b4daf3b9f663e472)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:23.817177 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:23.871278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '719297a86f0a4930b4daf3b9f663e472', 'request_id': 'req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.881125 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '719297a86f0a4930b4daf3b9f663e472', 'request_id': 'req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.887882 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '719297a86f0a4930b4daf3b9f663e472', 'request_id': 'req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.903026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] The token cache is being invalidate because project 719297a86f0a4930b4daf3b9f663e472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:33:23.903026 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.903807 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:23.905725 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1462/2888] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:33:23 2022] DELETE /identity/v3/projects/719297a86f0a4930b4daf3b9f663e472 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:33:23.910647 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:23.912431 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:23.927840 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e954a-9791-4c29-9d05-59b9e8c4bce9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:24.030901 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:24.031722 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:24.032100 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:24.032559 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] PATH_INFO: `/v3/users/8921309a07e24aca97786b9faead7d98` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:24.034772 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8921309a07e24aca97786b9faead7d98)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:24.037383 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:24.045104 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:33:24.103242 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8921309a07e24aca97786b9faead7d98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.103988 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8921309a07e24aca97786b9faead7d98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.112763 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8921309a07e24aca97786b9faead7d98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.113522 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8921309a07e24aca97786b9faead7d98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.130792 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8921309a07e24aca97786b9faead7d98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.146536 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2889] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:33:23 2022] DELETE /identity/v3/users/8921309a07e24aca97786b9faead7d98 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:33:24.489439 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:33:24.491632 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:33:24.512586 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c11b2-bcbc-4823-91ea-2aa6dd7a5166 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:33:24.570660 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:33:24.578615 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:33:24.579331 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:33:24.579966 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:33:24.580667 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] PATH_INFO: `/v3/projects/ab146e39de994e9db7b6e4e9b773d4c7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:33:24.582401 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab146e39de994e9db7b6e4e9b773d4c7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:33:24.584675 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:33:24.639887 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab146e39de994e9db7b6e4e9b773d4c7', 'request_id': 'req-6251a3f4-47a7-4135-b365-94c7ebdfa443'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.649871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab146e39de994e9db7b6e4e9b773d4c7', 'request_id': 'req-6251a3f4-47a7-4135-b365-94c7ebdfa443'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.657171 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab146e39de994e9db7b6e4e9b773d4c7', 'request_id': 'req-6251a3f4-47a7-4135-b365-94c7ebdfa443'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.670789 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] The token cache is being invalidate because project ab146e39de994e9db7b6e4e9b773d4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:33:24.671086 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.671854 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6251a3f4-47a7-4135-b365-94c7ebdfa443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:33:24.673507 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1463/2890] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:33:24 2022] DELETE /identity/v3/projects/ab146e39de994e9db7b6e4e9b773d4c7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:33:34.141903 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:33:34.667279 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:34:25.287910 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:34:25.289318 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:34:25.359334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1616-7574-4a66-9809-9f69b07cc27d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:34:25.418815 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:34:25.419503 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:34:25.419795 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:34:25.420078 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:34:25.421892 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:34:25.433748 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:34:25.435364 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:34:25.445787 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:34:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:34:35.441506 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:34:56.273748 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:34:56.274278 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:34:56.274388 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:34:56.294285 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] MFA Rules not processed for user `984ce8ca6aac41eab2c998c17d3820c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 05:34:56.295545 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:34:56.352571 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1464/2892] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 05:34:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 05:35:06.344523 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:36:45.092338 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:36:45.093873 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:45.150379 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d9b43-6192-46e6-b925-d8aa1d4788cf service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:36:45.154360 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:36:45.154977 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:36:45.155288 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:36:45.155603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:36:45.157591 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:45.168543 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:36:45.169997 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:36:45.179340 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:36:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:36:46.102396 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:36:46.113363 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fe90b-2d2a-486d-ac2e-e81fa5b80cd1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:36:46.117328 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:36:46.118326 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:36:46.118735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:36:46.118993 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:36:46.120847 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:46.134074 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:36:46.136023 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:36:46.147200 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1465/2894] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:36:46.252268 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:36:46.261631 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb277a-08d2-414d-9d7d-3168b3b09815 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:36:46.265407 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:36:46.266035 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:36:46.266323 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:36:46.266603 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:36:46.268411 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:46.345817 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:36:46.347305 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:36:46.358580 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:36:47.538787 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:36:47.540379 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:47.551781 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4b3f-f883-4af8-b0cb-90a2a4802f75 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:36:47.555541 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:36:47.556156 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:36:47.556443 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:36:47.556719 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:36:47.558412 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:47.572461 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:36:47.573884 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:36:47.585072 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1466/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:36:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:36:56.353498 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=60 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:36:57.580082 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=12 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:36:58.484554 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:36:58.493614 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66f6dc-8f26-4ed1-8fc0-67e08aef6410 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:36:58.497237 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac8ab3df29343c5bfa8928000f99b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:36:58.497847 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:36:58.498131 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:36:58.498403 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:36:58.500166 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:36:58.566038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:36:58.567526 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:36:58.577115 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 05:37:01.685242 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:37:01.686685 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:37:01.701579 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b8699-e4fe-4878-ab08-6bb23bc71449 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:37:01.706823 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:37:01.707876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:37:01.707876 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:37:01.708073 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:37:01.709928 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:37:01.724394 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:37:01.726192 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:37:01.750706 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1467/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:37:05.084740 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:37:05.086259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:37:05.099259 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073b224-5bdf-47fd-a56c-55280636b7a5 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:37:05.104097 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:37:05.104747 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:37:05.105038 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:37:05.105618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] PATH_INFO: `/v3/users/984ce8ca6aac41eab2c998c17d3820c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:37:05.106549 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=984ce8ca6aac41eab2c998c17d3820c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:37:05.108605 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:37:05.154696 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '984ce8ca6aac41eab2c998c17d3820c5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.155375 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '984ce8ca6aac41eab2c998c17d3820c5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.163071 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '984ce8ca6aac41eab2c998c17d3820c5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.163737 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '984ce8ca6aac41eab2c998c17d3820c5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.176989 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '984ce8ca6aac41eab2c998c17d3820c5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.190864 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2899] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:37:05 2022] DELETE /identity/v3/users/984ce8ca6aac41eab2c998c17d3820c5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:37:05.363260 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:37:05.376783 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440c93f-4ebd-4e78-afb2-6954dddce592 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:37:05.438709 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:37:05.439332 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:37:05.439654 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:37:05.439944 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] PATH_INFO: `/v3/projects/3d61aa3cfb59443d849c3d09d839cff5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:37:05.441124 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d61aa3cfb59443d849c3d09d839cff5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:37:05.443058 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:37:05.491267 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d61aa3cfb59443d849c3d09d839cff5', 'request_id': 'req-55f12b65-d37d-465b-b6e6-46e146927dba'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.498650 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d61aa3cfb59443d849c3d09d839cff5', 'request_id': 'req-55f12b65-d37d-465b-b6e6-46e146927dba'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.504871 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d61aa3cfb59443d849c3d09d839cff5', 'request_id': 'req-55f12b65-d37d-465b-b6e6-46e146927dba'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.516664 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] The token cache is being invalidate because project 3d61aa3cfb59443d849c3d09d839cff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:37:05.517495 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.518168 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.519735 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1468/2900] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:37:05 2022] DELETE /identity/v3/projects/3d61aa3cfb59443d849c3d09d839cff5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:37:05.523789 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:37:05.525232 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:37:05.539164 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0f544-4da0-4162-9479-56fe8ffda9e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:37:05.594056 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:37:05.595275 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:37:05.595586 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:37:05.595861 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] PATH_INFO: `/v3/users/89202d24880044529b5fc5560b1b184c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:37:05.597033 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89202d24880044529b5fc5560b1b184c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:37:05.598954 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:37:05.604220 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:37:05.658944 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89202d24880044529b5fc5560b1b184c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.659649 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89202d24880044529b5fc5560b1b184c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.666681 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89202d24880044529b5fc5560b1b184c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.667334 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89202d24880044529b5fc5560b1b184c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.679912 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89202d24880044529b5fc5560b1b184c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:05.693135 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2901] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 05:37:05 2022] DELETE /identity/v3/users/89202d24880044529b5fc5560b1b184c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 05:37:05.889852 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:37:05.891987 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:37:05.906202 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f12b65-d37d-465b-b6e6-46e146927dba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:37:05.967936 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b2847a9d0472682c985005f47d70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3340905a228b4864923e8ce0f5cfd8a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:37:05.968512 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:37:05.968802 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:37:05.969110 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] PATH_INFO: `/v3/projects/9bf85229f25c4bd7b45355082e2fc952` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:37:05.970340 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf85229f25c4bd7b45355082e2fc952)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:37:05.972357 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:37:05.994135 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:37:06.022437 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf85229f25c4bd7b45355082e2fc952', 'request_id': 'req-82674198-404d-4897-bf5d-ad1add5ee1d7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:06.030652 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf85229f25c4bd7b45355082e2fc952', 'request_id': 'req-82674198-404d-4897-bf5d-ad1add5ee1d7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:06.036464 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf85229f25c4bd7b45355082e2fc952', 'request_id': 'req-82674198-404d-4897-bf5d-ad1add5ee1d7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:06.049036 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] The token cache is being invalidate because project 9bf85229f25c4bd7b45355082e2fc952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 05:37:06.049330 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:06.049997 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-82674198-404d-4897-bf5d-ad1add5ee1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 05:37:06.051557 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1469/2902] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 05:37:05 2022] DELETE /identity/v3/projects/9bf85229f25c4bd7b45355082e2fc952 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 05:37:15.689445 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:37:16.045651 dv-cinder-851279-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 05:39:28.275627 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 05:39:28.277075 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:39:28.425618 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd5a09-17dc-4246-9971-5dcfc7cc0026 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 05:39:28.487410 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5df6c21c2ed44bf0b07cd16454b3e0ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd00b15fba514550924237d11784db71', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 05:39:28.487947 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 05:39:28.488233 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 05:39:28.488512 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 05:39:28.490264 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 05:39:28.502995 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 05:39:28.504629 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc021f-7cd9-447b-94d9-2876cb852d0b service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 05:39:28.515645 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 05:39:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 05:39:38.510482 dv-cinder-851279-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}